Build, test, and leverage technological solutions to combat systemic racism  Get involved

Tutorial

Encrypt and monitor the usage of objects stored in IBM Cloud Object Storage

Secure your data in IBM Cloud Object Storage and track the usage

Save Like

By Balaji Kadambi
Published October 7, 2020

Learning objectives

IBM Cloud Object Storage is a widely used service for storing documents. When documents have sensitive and confidential information, you need to encrypt the contents and set the access policies.

In this tutorial, you learn how to:

  • Use IBM Key Protect for IBM Cloud to encrypt objects stored in IBM Cloud Object Storage
  • Create a bucket programmatically with encryption using the IBM Cloud Object Storage SDK for Java
  • Upload objects in the bucket
  • Access the objects in the bucket
  • Monitor the usage of the bucket for read and write using the IBM Cloud Activity Tracker with LogDNA

Prerequisites

Estimated time

Completing this tutorial should take about 45 minutes.

Steps

1. Log in to IBM Cloud

Open a terminal and run the following command to log in to IBM Cloud:

ibmcloud login

For single sign-on, run the following command and log in to IBM Cloud:

ibmcloud login --sso

Set the organization and space:

ibmcloud target --cf

2. Set up IBM Cloud Object storage

Create an instance of IBM Cloud Object Storage

Run the following command to create an instance of IBM Cloud Object Storage with the name my-storage.

ibmcloud resource service-instance-create my-storage cloud-object-storage lite global

You should see the following output after successfully creating the instance:

Creating service instance my-storage in resource group default of account xxxx Account as xxxx...
OK
Service instance my-storage was created.

Name:             my-storage   
ID:               crn:v1:bluemix:public:cloud-object-storage:global:a/xxxx:xxxx::   
GUID:             xxxx   
Location:         global   
State:            active   
Type:             service_instance   
Sub Type:            
Allow Cleanup:    false   
Locked:           false   
Created at:       2020-08-19T09:45:43Z   
Updated at:       2020-08-19T09:45:43Z   
Last Operation:                   
                  Status    create succeeded      
                  Message   Completed create instance operation

Make a note of the GUID in the output. You need it to create an authorization policy. Also, note the ID. You need it later to access IBM Cloud Object Storage from your Java program.

Note: This command can give an error if there is an existing instance with the Lite plan already created. In such a scenario, you can reuse the existing instance. Run the ibmcloud resource service-instance [instance name] command to get the ID and GUID of the existing instance.

Create a service key to access IBM Cloud Object Storage

Run this command to generate a service key for IBM Cloud Object Storage:

ibmcloud resource service-key-create skey --instance-id [GUID of Cloud Object Storage]

You should see the following output:

Creating service key of service instance xxxx under account xxxx Account as xxxx...
OK
Service key crn:v1:bluemix:public:cloud-object-storage:global:a/xxxx:xxxxx:resource-key:xxxx was created.

Name:          skey   
ID:            crn:v1:bluemix:public:cloud-object-storage:global:a/xxxx:resource-key:xxxx   
Created At:    Thu Aug 20 09:51:51 UTC 2020   
State:         active   
Credentials:                                   
               apikey:                   xxxx      
               endpoints:                https://control.cloud-object-storage.cloud.ibm.com/v2/endpoints      
               iam_apikey_description:   Auto-generated for key xxxx      
               iam_apikey_name:          skey      
               iam_role_crn:             crn:v1:bluemix:public:iam::::serviceRole:Writer      
               iam_serviceid_crn:        crn:v1:bluemix:public:iam-identity::a/xxxx::serviceid:ServiceId-xxxx      
               resource_instance_id:     crn:v1:bluemix:public:cloud-object-storage:global:a/xxxx:xxxx::

Make a note of the credentials. You need it later to access IBM Cloud Object Storage from your Java program.

3. Set up Key Protect

Create an instance of Key Protect

Run the following command to create an instance of Key Protect after specifying the region (for example, us-south):

ibmcloud resource service-instance-create my-key kms tiered-pricing [region]

You should see following output:

Creating service instance my-key in resource group default of account xxxx Account as xxxx...
OK
Service instance my-key was created.

Name:             my-key   
ID:               crn:v1:bluemix:public:kms:au-syd:a/xxxx:xxxx::   
GUID:             xxxx   
Location:         us-south   
State:            active   
Type:             service_instance   
Sub Type:         kms   
Allow Cleanup:    false   
Locked:           false   
Created at:       2020-08-19T10:03:28Z   
Updated at:       2020-08-19T10:03:28Z   
Last Operation:                   
                  Status    create succeeded      
                  Message   Completed create instance operation

Make a note of the GUID in the output; you need it to create an authorization policy.

Create a root key

Run the following command to create a root key after specifying the region (for example, us-south):

ibmcloud kp region-set [region]
ibmcloud kp key create my-root-key -i [GUID of Key Protect instance]

Note: The region set here is the location (for example, us-south) where Key Protect was created.

You should see the following message:

Creating key: 'my-root-key', in instance: 'xxxx'...
OK
Key ID                                 Key Name   
xxxx                                   my-root-key

Create an authorization

Run this command to grant access of the Key Protect instance to the Cloud Object Storage instance:

ibmcloud iam authorization-policy-create cloud-object-storage kms Reader --source-service-instance-id [GUID of Cloud Object Storage Instance]  --target-service-instance-id [GUID of Key-Protect instance]

Replace the GUID that we noted earlier in the previous command for both the IBM Cloud Object Storage and Key Protect instances.

You should see the following output:

Creating authorization policy under account xxxx as xxxx...
OK
Authorization policy xxxx was created.

ID:                        xxxx   
Source service name:       cloud-object-storage   
Source service instance:   xxxx   
Target service name:       kms   
Target service instance:   xxxx  
Roles:                     Reader

Create an instance of LogDNA with Activity Tracker

Run this command to create an instance of LogDNA with Activity Tracker after specifying the region (for example, us-south):

ibmcloud resource service-instance-create my-logs logdnaat lite [region]

You should see the following output:

Creating service instance my-logs in resource group default of account xxxx Account as xxxx...
OK
Service instance my-logs was created.

Name:             my-logs   
ID:               crn:v1:bluemix:public:logdnaat:au-syd:a/xxxx:xxxx::   
GUID:             xxxx   
Location:         us-south   
State:            active   
Type:             service_instance   
Sub Type:            
Allow Cleanup:    false   
Locked:           false   
Created at:       2020-08-19T13:22:35Z   
Updated at:       2020-08-19T13:22:35Z   
Last Operation:                   
                  Status    create succeeded      
                  Message   Completed create instance operation

Make a note of the ID. You need it later to configure the LogDNA instance for IBM Cloud Object Storage from your Java program.

4. Create an instance of LogDNA with Activity Tracker

Go to the IBM Cloud dashboard and view the services created.

services

Go to Manage > Access on the IBM Cloud dashboard.

manage

Click Authorizations on the menu. You should see the authorization created earlier.

authorization

5. View the services created on the IBM Cloud dashboard

Get the root key CRN

Go to the my-key service on IBM Cloud Dashboard:

openkey

Click Manage Keys and select the menu item View CRN.

viewcrn

Make a note of the displayed root key CRN. You need to configure the Java program to create an encrypted bucket.

Open the LogDNA dashboard

Open the LogDNA dashboard from the IBM Cloud console. Once you run the Java programs, you can come back to this console to view the logs.

logdnadb

6. Set up the Java client

Clone the GitHub repo

Run the following command to clone the Github repo:

git clone https://github.com/IBM/object-storage-encryption

This creates the folder object-storage-encryption. This cloned repo folder has the Java code to:

  • Create a new encrypted bucket (CreateBucket.java)
  • Upload an object to the bucket (UploadObject.java)
  • Configure LogDNA on the bucket instance (ConfigureLogging.java)
  • Download an object (ReadObject.java)

The code has been built using the IBM Cloud Object Storage SDK.

Edit the configuration

Open the Constants.java file under the cloned folder object-storage-encryption/src/main/java/com/example. Complete the configuration as follows:

You can modify the other fields based on the encryption algorithm needed, region, and type of the bucket you create in IBM Cloud Object Storage.

public static String COS_KP_ALGORITHM = "AES256";
public static String COS_KP_ROOTKEY_CRN = "crn:v1:bluemix:public:kms:us-south:a/xxx:key:xxx";
public static String COS_ENDPOINT = "https://s3.us-south.cloud-object-storage.appdomain.cloud"; 
public static String COS_API_KEY_ID = "xxxx"; 
public static String COS_AUTH_ENDPOINT = "https://iam.cloud.ibm.com/identity/token";
public static String COS_SERVICE_CRN = "crn:v1:bluemix:public:cloud-object-storage:global:a/xxxx::"; 
public static String COS_BUCKET_LOCATION = "us-south-smart"; 
public static String BUCKET_NAME = "my-bucket" + "-UID";// add an unique identifier UID
public static String AT_CRN = "crn:v1:bluemix:public:logdnaat:us-south:a/xxxx::";

Build the JAR file

Open a terminal. Go to the folder object-storage-encryption. Run the following commands under the cloned repo folder object-storage-encryption:

mvn install
cp target/object_storage_key_protect-0.0.1-SNAPSHOT-jar-with-dependencies.jar oskp.jar

7. Run the Java client

Create a bucket with encryption enabled

Run the command to create a bucket:

$ java -cp oskp.jar com.example.CreateBucket

You should see the following output:

Creating new encrypted bucket: my-bucket-bkad-27aug
Bucket: my-bucket-bkad-27aug created!

Configure logging with LogDNA

Run the command to configure logging for the bucket:

java -cp oskp.jar com.example.ConfigureLogging

You should see the following output:

Aug 27, 2020 2:11:19 PM okhttp3.internal.platform.Platform log
INFO: --> POST https://iam.bluemix.net/identity/token http/1.1 (135-byte body)
Aug 27, 2020 2:11:21 PM okhttp3.internal.platform.Platform log
INFO: <-- 200 OK https://iam.bluemix.net/identity/token (1355ms, 1687-byte body)
Aug 27, 2020 2:11:23 PM okhttp3.internal.platform.Platform log
INFO: --> PATCH https://config.cloud-object-storage.cloud.ibm.com/v1/b/my-bucket-bkad-27aug http/1.1 (211-byte body)
Aug 27, 2020 2:11:24 PM okhttp3.internal.platform.Platform log
INFO: <-- 200 OK https://config.cloud-object-storage.cloud.ibm.com/v1/b/my-bucket-bkad-27aug (715ms, unknown-length body)
Aug 27, 2020 2:11:24 PM okhttp3.internal.platform.Platform log
INFO: --> GET https://config.cloud-object-storage.cloud.ibm.com/v1/b/my-bucket-bkad-27aug http/1.1
Aug 27, 2020 2:11:24 PM okhttp3.internal.platform.Platform log
INFO: <-- 200 OK https://config.cloud-object-storage.cloud.ibm.com/v1/b/my-bucket-bkad-27aug (583ms, 726-byte body)
{
  "name": "my-bucket-bkad-27aug",
  "crn": "crn:v1:bluemix:public:cloud-object-storage:global:a/xxxx:bucket:my-bucket-bkad-27aug",
  "service_instance_id": "xxxx",
  "service_instance_crn": "crn:v1:bluemix:public:cloud-object-storage:global:a/xxxx:xxxx::",
  "time_created": "2020-08-27T14:01:39.325",
  "time_updated": "2020-08-27T14:11:24.015",
  "object_count": 0,
  "bytes_used": 0,
  "activity_tracking": {
    "read_data_events": true,
    "write_data_events": true,
    "activity_tracker_crn": "crn:v1:bluemix:public:logdnaat:us-south:a/xxxx:xxxx::"
  }
}

Upload object

Run the command to upload a file:

$ java -cp oskp.jar com.example.UploadObject

A text test content is uploaded as a file test.txt into the bucket you created in the earlier step. You should see the following output:

Uploaded file test.txt

Download object

Run the command to download the file “test.txt”:

java -cp oskp.jar com.example.ReadObject

You should see the following output:

test content

8. View usage details on LogDNA

Switch to the open LogDNA dashboard from the IBM Cloud console.

logdnalogs

You should see two events for Key Protect and two events for Cloud Object Storage. You can analyze the logs for security events.

Summary

In this tutorial, you created an encrypted bucket on IBM Cloud Object Storage programmatically. You viewed the object upload and download events on the LogDNA. I hope you found the tutorial useful!

Legend
Tutorial

Create a custom Appsody stack with template for IBM Cloud Object Storage operations

July 22, 2020
Code Pattern

Serverless image processing with Cloud Object Storage

July 10, 2019
Article

Modernizing the Weather Underground website with cloud object storage

April 26, 2019