Overview

Skill Level: Beginner

Whether you are new to IBM Cloud ObjectStorage or a seasoned pro, you will want to make sure you know how to access a bucket like you might a remote filesystem.

Ingredients

Access to an IBM Cloud Object Storage (COS) bucket

A Linux based system

s3fs-fuse can be used to mount an IBM Bluemix Cloud ObjectStorage (COS) S3 API bucket as a file system on your local Linux or Mac system. Once mounted as a local file system, you can use other system tools to interact with your bucket. If you don’t yet have a COS account, see Ordering Storage

Step-by-step

  1. Building the source

    The s3fs-fuse GitHub page has all the details on building and installing the package. However, the quick and easy steps on an Ubuntu 14.04 VM are as follows:

    • Run these commands to get the most recent versions of the prerequisite libraries
    sudo apt-get update
    sudo apt-get install automake autotools-dev g++ git libcurl4-openssl-dev libfuse-dev libssl-dev libxml2-dev make pkg-config
    git clone https://github.com/s3fs-fuse/s3fs-fuse.git

    • Clone the latest source from github
    git clone https://github.com/s3fs-fuse/s3fs-fuse.git
    • Build and install the library
    cd s3fs-fuse
    ./autogen.sh
    ./configure
    make
    sudo make install

    That’s all there is to it. You can now start using s3fs.

    Note: The above command is slightly different from the instructions provided by s3fs-fuse. Testing has shown that replacing libcurl4-gnutls-dev with libcurl4-openssl-dev shows improved performance.

  2. Configuration and mounting

    In order to configure s3fs-fuse, you need your access key id, your secret access key, the name of the bucket you want to mount, and the endpoint for the 

    If you are using the Infrastructure variation of Cloud Object Storage (i.e. softlayer), you can get these values from the ObjectStorage section in the Control Portal.

    If you are using the IBM Cloud variation of Cloud Object Storage, you can follow these instructions to create HMAC credentials.

    There are several different endpoints for IBM Cloud Object Storage. If you don’t know which end-point your bucket is in, have a look in your dashboard.

    • Now that you have your credentials, you are ready to configure s3fs-fuse and mount your bucket. From the command line of your Linux system, add your credentials to a configuration file and fix the permissions to limit access. Substitue your values for <Access_Key_ID> and <Secret_Access_Key> in the below command.
    echo "<Access_Key_ID>:<Secret_Access_Key>" > $HOME/.cos_creds
    chmod 600 $HOME/.cos_creds
    • Create a directory where you can mount your bucket. Typically, this is done in the /mnt directory on Linux.
    sudo mkdir /mnt/mybucket
    • Mount the bucking using the command below. Be sure to use the endpoint that corresponds to the location of your bucket. In this example, I will be using the US East (us-east) Regional service, public endpoint, and the bucket “mpc-mdms-test-east”
    sudo s3fs mpc-mdms-test-east /mnt/mybucket -o passwd_file=$HOME/.cos_creds -o sigv2 -o use_path_request_style -o url=https://s3.us-east.objectstorage.softlayer.net

    That’s it. The contents of your bucket should now be available in /mnt/mybucket.

    Creating, editing, deleting files you see in this directory will be reflected in your COS bucket. Because the files are really objects in COS, some operations will take additional time for network transfers to complete.

    If you have problems connecting or want the bucket to auto mount on reboots, the s3fs-fuse instructions have more details for you. The s3fs-fuse Wiki is another good source of information.

     

Join The Discussion