Use IBM Cloud Hyper Protect Crypto Services to encrypt VMware disks

IBM Cloud offers integrated VMware solutions. Each virtual machine you stand up has storage coupled to it, which you may want to encrypt. These VMs may host applications and data that contain sensitive information, so you would need to lock it down.

You can encrypt this storage via highly secure, industry-standard algorithms. But this can lead to a key management concern: Where do you keep the keys, and how do you secure them? You can now configure a tight integration between IBM Cloud Hyper Protect Crypto Services (HPCS) and VMware on IBM Cloud. This tutorial shows you how to set this up to ensure that your most sensitive data is protected.

HPCS allows for secure key generation and storage, and takes advantage of an industry-leading hardware security module (HSM). This is the only public-cloud HSM that offers FIPS 140-2 level 4 data protection, which means that it’s highly tamper resistant. Store your keys here, and you can be sure that they’re kept safe from hackers — and even from IBM. No one but you can read them.

Learning objectives

This tutorial will show you how to integrate IBM Cloud Hyper Protect Crypto Services (HPCS) with VMware solutions in IBM Cloud. You’ll learn how to configure IBM Cloud Hyper Protect Crypto Services with a master key, which can be split across multiple systems or people in your organisation, and is known only to you. With this, taking advantage of envelope encryption, your VMware disks can be encrypted while the keys are stored securely. As part of this, you’ll learn how to integrate HPCS with VMware using a KMIP adapter.

Prerequisites

To complete this tutorial, you’ll need:

  • An IBM Cloud account
  • A VMware environment in IBM Cloud
  • An HPCS instance
  • The IBM Cloud CLI, with TKE plug-in (this tutorial tells you how)

Estimated time

It should take you about 1 – 2 hours to complete this tutorial.

Components of the solution

To complete this solution, you’ll need to follow these three general steps:

  1. Create keys in HPCS
  2. Create a Key Management Interoperability Protocol (KMIP) instance, with a mutual authentication handshake between it and your HPCS instance
  3. Connect your KMIP instance to your VMware environment, again after handshaking for authentication.

Let’s start with the HPCS key creation process:

1

Create keys in HPCS

Create your HPCS instance in IBM Cloud. Once you’ve done that, you need to move to the CLI to create the keys. Make a note of your instance endpoint, for example:

https://api.us-south.hs-crypto.cloud.ibm.com:8069

To get started with the CLI install, you’ll first need the [ibmcloud CLI][cli]. Once that’s installed, login with ibmcloud login. Then, make sure that you are logged in to the correct API endpoint:

ibmcloud target -r us-south

Next, you need to install the Trusted Key Entry plugin, so you can securely add keys to your HSM instance:

ibmcloud plugin install tke

You should then create some master key part files and signature key part files. These need to be stored in a directory, which needs to be named in the variable CLOUDTKEFILES. For example:

mkdir "$HOME/tke"
export CLOUDTKEFILES="$HOME/tke"

Your HPCS instance is, or can be, split into crypto units, which can be interchanged and are ideal for load balancing and high availability. They start off in a cleared imprint mode state.

Confirm what crypto units you have:

ibmcloud tke cryptounits

Note the SELECTED column in the output of that command, for example:

CRYPTO UNIT NUM   SELECTED   LOCATION
1                 false      [us-south].[AZ2-CS2].[03].[04]

The crypto units that are selected are the ones that your future admin operations will be performed on. So you need to select this crypto unit in order to operate on it:

ibmcloud tke cryptounit-add

At the prompt, enter 1 to select that crypto unit. This confirms the selection in its output:

CRYPTO UNIT NUM   SELECTED   LOCATION
1                 true       [us-south].[AZ2-CS2].[03].[04]

Now that you’ve selected the crypto unit, you can perform operations against it. The next thing to do is to create and load a master key into a master key register. To do this, in turn, you need to create a signature key. With this, you can sign the operation of creating the master key.

Create a signature key:

ibmcloud tke sigkey-add

That will then ask for a username and password to protect it with: I chose admin / passw0rd for this demo and tutorial. Then, select that as the signature key to perform future signing operations with:

ibmcloud tke sigkey-sel

At the prompt, choose key 1 and enter the password you just used.

Now you need to add this admin user (and signature key) to the crypto unit so it can perform operations. To do this:

ibmcloud tke cryptounit-admin-add

Enter 1 to choose the admin user you just created, and enter the key’s password again.

Next, you need to exit imprint mode as the initial setup is complete:

ibmcloud tke cryptounit-exit-impr

You’ll then be asked for the admin (signature key’s) password one more time. Once you’re out of imprint mode, you can create a set of master key parts.


The general idea is to create multiple parts of a master key, to be split across different people for added security — similar to a nuclear submarine, where multiple individuals would need to use their keys at the same time.

Let’s create a master key part:

ibmcloud tke mk-add --random

I gave my key a description of key 1, and a password of passw0rd. Again, this is just a simple example for demonstration purposes; you should always choose a highly secure password and store it carefully.

KEYNUM   DESCRIPTION   VERIFICATION PATTERN
1        key 1         2857093ecf5cff1fffd06fd9268c997c
                       42b73beacd2d850064d1a6f509771ea2

You need at least two master key parts, so repeat this process again:

ibmcloud tke mk-add --random

You should then see the following:

KEYNUM   DESCRIPTION   VERIFICATION PATTERN
1        key 1         2857093ecf5cff1fffd06fd9268c997c
                       42b73beacd2d850064d1a6f509771ea2
2        key 2         fc5bba79b04baf8220ab3eaebb76d5c6
                       ca96a3a3721761e46245ade12119b908

You’ll notice that the two master key parts and the signature key are on your current machine, in the directory CLOUDTKEFILES. These need to be on the same machine to add the master key parts to an HSM crypto unit. To do this, run:

ibmcloud tke cryptounit-mk-load

Entering 1 2 at the prompt. Passwords for all three keys will now be required. For this tutorial, I’ve created them all, but in practice these will likely be different passwords, known to different people, and can be stored on different systems.

Now, you need to commit this master key register:

ibmcloud tke cryptounit-mk-commit

Since this is an operation against the crypto unit, you will then be asked for the password for the signature key for the administrator with access to that crypto unit. Now you should see a fully-committed key:

NEW MASTER KEY REGISTER
SERVICE INSTANCE: 8b1eb136-b02d-4e71-b715-22f7169f8fef
CRYPTO UNIT NUM   STATUS           VERIFICATION PATTERN
1                 Full Committed   d2e7f8abd41e8042801da385d14c2003
                                   d4fd42c1c37bde4b672ac6fcff42cff9

There are two registers for master keys in the crypto unit — one for a new key and one for the current key. To activate this master key, it needs to become the current key:

ibmcloud tke cryptounit-mk-setimm

Type y when prompted. Again, this operation will require the permission of the administrator, and thus the password to their signature key.

You’ll see confirmation of this key move, with the current master key register populated:

NEW MASTER KEY REGISTER
SERVICE INSTANCE: 8b1eb136-b02d-4e71-b715-22f7169f8fef
CRYPTO UNIT NUM   STATUS   VERIFICATION PATTERN
1                 Empty    00000000000000000000000000000000
                           00000000000000000000000000000000


CURRENT MASTER KEY REGISTER
SERVICE INSTANCE: 8b1eb136-b02d-4e71-b715-22f7169f8fef
CRYPTO UNIT NUM   STATUS   VERIFICATION PATTERN
1                 Valid    d2e7f8abd41e8042801da385d14c2003
                           d4fd42c1c37bde4b672ac6fcff42cff9

Viewing the manage tab of the HPCS instance in IBM Cloud, you should see that the HSM master key has been configured:

HSM master key


Now let’s create a root key. HPCS uses the recommended envelope encryption mechanism of taking a key that’s used to encrypt data, the data encryption key or DEK, and encrypting the key itself with a root key. This root key never leaves the HSM.

In a web browser go to your IBM Cloud resources view, and find your HPCS instance.

Click the add key button and then generate a new root key. I’ve called mine root1.

Perform this operation again to create a standard key, called vmware-dek.

The HPCS part of the setup is now complete. Next you need to install and configure a KMIP adapter in IBM Cloud.

2

Create a KMIP instance in IBM Cloud

Your VMware server has storage attached to it that you’ll want to encrypt using the keys stored in the HPCS instance that you’ve just created. VMware doesn’t talk via the same protocols as HPCS, though, so you need to set up an interfacing service, the VMware Key Management Interoperability Protocol (KMIP).

Ensure that you have an IaaS account and that your IaaS account has virtual routing and forwarding (VRF) and service endpoints enabled.

  1. Go to the KMIP adapter page on IBM Cloud.

  2. Generate an API key:

    ibmcloud iam api-key-create MyHMKey -d "KMIP adapter"
    

    And copy the “API key” entry on to the above page.

  3. Retrieve your key manager instance; mine shows hpvs-vmware.

  4. Under the customer root key dropdown, choose the root key created… here, it’s root1.

This will take you to a VMware Solutions page. Under KMIP for VMware on IBM Cloud Instances, you’ll see your HPCS instance installing.

That should now be all configured. Finally, you’re ready to couple this KMIP instance to your VMware in IBM Cloud installation.

  1. Click the link to your new KMIP adapter. It will confirm the name of your HPCS instance and your customer root key.

  2. Download the KMIP server CA certificate and the KMIP server certificate. Note the endpoints, for example:

    kmip-1.private.us-south.vmware-solutions.cloud.ibm.com:5696
    kmip-2.private.us-south.vmware-solutions.cloud.ibm.com:5696
    
3

Connect KMIP to VMware in IBM Cloud

Now you want to couple the KMIP adapter to your VMware vCenter or vSphere instance. First, you need to configure the VPN so you can access the VMware console:

  1. Connect via a MotionPro Plus VPN client. I grabbed the 9.4.0.x branch after following this blog post.

  2. The VPN client needs to know the endpoint to connect to. On your [VMware Solutions Resources][vm-resource] page, choose the instance you want to configure your KMIP adapter with.

  3. Note where it’s located — for me, it’s DAL13 (Dallas).

  4. Find the corresponding SoftLayer VPN endpoint. For example, for DAL13 it’s:

    vpn.dal.softlayer.com
    
  5. Connect your VPN to site address [endpoint]:443 with the username and password for your VPN infrastructure account.

Now you should be able to connect to the VMware console on the private network.

  1. Edit the /etc/hosts or equivalent file on your local system to map the vCenter/PSC IP IP address to the vCenter/PSC FQDN domain. For example:

    10.208.85.196  vcenter.vcs-scott.example.com
    10.208.85.196  vcenter-vcs-scott.vcs-scott.example.com
    
  2. On your [VMware Solutions Resources][vm-resource] page, click the button to open the console, or navigate to the domain given in the vCenter/PSC FQDN field.

  3. Launch the client. I chose the HMTL5 variant.

  4. At the username and password prompt, from the VMware resources page, find the combination given in the vCenter/PSC ADMIN field. You’ll need to click the eye icon to reveal the password; the username is shown before the /.

  5. Click Menu -> Global Inventory Lists -> vCenter Servers -> Configuration -> Key Management Servers -> Add.

  6. Choose to create a new cluster and give it a name, along with the server name (name of the KMIP adapter).

  7. Get the hostname and port from your KMIP adapter. If you need to specify proxy details, do so, but I’ve left it blank.

  8. Click Trust to make vCenter trust KMS.

  9. For the populated key management server entry, click the drop-down icon to see how far the chain of trust has gone. Click Make KMS Trust vCenter. Choose vCenter Certificate. This will generate a root CA certificate for you, which needs to be passed to the KMS. Click Copy.

  10. Back in the KMIP adapter config page, under Client SSL Certificates, click the Add button. Give it a name, then click Add.

  11. Go back to the vSphere Client, and click Done. If you click the refresh symbol in the client, you’ll see that trust has now been established between the vSphere environment and the KMIP adapter.

Now you can make use of this integration by creating a new virtual machine with encrypted storage, with its key protected with HPCS.

Create a new VM with encrypted storage

From the vCenter Servers page in the vSphere Client:

  1. Right-click the data center under the hostname in vSphere client on the left.
  2. Choose New Virtual Machine…, and Create a new virtual machine.
  3. Choose Next, and give it a name.
  4. Choose the defaults until you get to VM Storage Policy, and choose VM Encryption Policy in the dropdown. (We chose Management share for compatible storage; we’re going to use a 64-bit Ubuntu Linux image.)
  5. Click browse under New Network to choose SDDC-DPortGroup-Mgmt (in our case).
  6. Choose a datastore ISO image to define the VM as, and click the connect box.

Once you’ve done this, choose the new VM on the left-hand side. Under VM Hardware, it should say that the disk and configuration files are encrypted.

If you click the task in the Recent Tasks pane and choose the VM, one of the log items will be for CreateVM, indicating that it’s an encrypt disk operation that uses the KMIP adapter you defined previously. If you go back to the Manage page for the HPCS instance, you’ll see the new key in use.

And that’s all there is to it. Once the HPCS and KMIP parts of this tutorial have been completed, the actual setup and use of this connection is quite fast, and your keys are now protected by the industry’s best hardware security module.

Summary

IBM Cloud Hyper Protect Crypto Services allows for secure key generation and storage, and takes advantage of an industry-leading Hardware Security Module (HSM). This is the only public cloud HSM that offers FIPS 140-2 level 4 data protection, which means that it’s highly tamper-resistant. Store your keys here, and you can be sure they’re kept safe from hackers, and even from IBM. No one but you can read them.

With the integration between this service and the VMware solutions in IBM Cloud, you can take advantage of this best-of-class technology to encrypt VM disks across your VMware estate. For more information, check out the IBM Cloud Hyper Protect Crypto Services demos.

Chris Poole