Provision, bind, and use services in a cloud-native way

Overview

Applications often rely on a number of services to do their job. Called backing services, these can include databases, search engines, and AI engines. The twelve-factor methodology for app development states that it is crucial to provision and bind a backing service as part of your application configuration (described in Factor 3 and Factor 4). In addition, it is important that you implement a secure way to retrieve connection information, such as credentials. This tutorial shows you how to manage backing services as code using the IBM Cloud Operator and the Red Hat Service Binding Operator to provision, bind, and use services in a cloud-native way.

In this tutorial, you deploy an application that consumes a backing service, where all the configuration parameters to provision and bind the service are also part of the application configuration. The IBM Cloud Operator interacts between the Kubernetes cluster (here OpenShift) and the IBM Cloud services. Various Kubernetes resources are created for each requested cloud service. The Red Hat Service Binding Operator takes over the retrieval of the credentials and injects them into the containerized app.

The following figure visualizes the solution in a Red Hat OpenShift cluster with the related components and Kubernetes resources:

Figure 1. Sample payment workflow

Prerequisites

Estimated time

Completing this tutorial should take about 45 minutes.

Checkout git resources

You can download helpful scripts and Kubernetes resources from the GitHub repository.

Clone the repository by running this command:

git clone https://github.com/IBM/cloud-service-management-tutorial

The scripts directory holds all of the relevant files, which you use during the tutorial.

IBM Cloud Operator

IBM Cloud Operator provisions and binds services from the IBM Cloud catalog.

The IBM Cloud Operator supports the following use cases:

  • Provisioning a service from the IBM Cloud catalog
  • Binding a service instance and storing the credentials as a secret in the namespace

For these use cases, you need to grant the IBM Cloud Operator access to the IBM Cloud service catalog. To do this, you generate an API key from a service ID. This grants permission to the service ID and is not linked to a personal user account. You can also set the right permissions to the service ID for each backing service that you want to use.

Create the API key

To generate the API key, use the ibmcloud CLI with IBM Cloud Shell.

The following listing provides all the commands to create a service ID serviceid-ico, set some permissions, and generate the API key. Adjust the commands to your needs and with your unique identifiers. When you set permissions, consider following the principle of least privilege, which recommends giving users permission to only what is needed for their job functions.

# List the available regions, the region name is used later for adding a policy to service ID
$ ibmcloud regions
Listing regions...

Name            Display name   
au-syd          Sydney   
in-che          Chennai   
jp-osa          Osaka   
jp-tok          Tokyo   
kr-seo          Seoul   
eu-de           Frankfurt   
eu-gb           London   
ca-tor          Toronto   
us-south        Dallas   
us-south-test   Dallas Test   
us-east         Washington DC   
br-sao          Sao Paolo   

# Display the resource groups. Needed for adding a policy to service ID
$ ibmcloud resource groups
Retrieving all resource groups under account ... as Hafid...
OK
Name      ID                                 Default Group   State   
Default   ...                                true            ACTIVE   


# Create a new service ID with the name "serviceid-ico"
$ ibmcloud iam service-id-create serviceid-ico -d service-ID-for-ibm-cloud-operator
...

# Add policy Administrator to the created service ID in specific resource group and region
$ ibmcloud iam service-policy-create serviceid-ico --roles Administrator --resource-group-name Default --region eu-de

# Displays the policies to the created service ID (here: serviceid-ico). Helpful to review or update the current configuration
$ ibmcloud iam service-policies serviceid-ico

# In case you to update the policy use this command with the Policy ID from the previous output
$ ibmcloud iam service-policy-update serviceid-ico 5be... --roles Manager --resource-group-name Default
...

# After setting permissions, create an API key with name "apikey-ico" for service ID "serviceid-ico"
$ ibmcloud iam service-api-key-create apikey-ico serviceid-ico -d api-key-for-ibm-cloud-operator
Creating API key apikey-ico of service ID serviceid-ico under account ... as Hafid...
OK
Service ID API key apikey-ico is created

Please preserve the API key! It cannot be retrieved after it's created.

ID            ApiKey-25f....   
Name          apikey-ico   
Description   api-key-for-ibm-cloud-operator   
Created At    2020-12-06T10:09+0000   
API Key       N....   
Locked        false

Remember to preserve the API key!

Prepare and install the IBM Cloud Operator

To interact with the IBM Cloud catalog, the IBM Cloud Operator needs the API key of the service ID. You should place this sensitive information in a separate namespace that is restricted to a small amount of operation/admin colleagues, not in the same namespace where you are going to deploy the custom application.

IBM Cloud Operator supports this separation, and you can configure secrets in a separate namespace. You can also have separate API keys for individual namespaces.

In each (custom) namespace, the IBM Cloud Operator expects a Secret and ConfigMap prefixed with the individual namespace. The Secret contains the API key and region. The ConfigMap holds the default values for region, user ID, and resource group. You can replace these values later for a specific service provisioning (see the documentation for details on how to overwrite those default values).

The operator team provides an install script to configure and install the operator. In case you do not have the IBM Cloud CLI on your machine, use the script scripts/ico/prepare_ico.sh, which prepares the configuration if you have all of the following parameters:

  • API Key
  • Resource group and ID
  • Service ID
  • Region

This generates the namespace ibmcloud-operator-configs and the initial API keys for one (custom) namespace ico-test. The following listing contains an exemplary command execution with the output. Replace the parameters accordingly.

$ cd scripts/ico
$ ./prepare_ico.sh <your-API-Key> <Resource-Group-Name> <Resource-Group-ID> <Service-ID-Name>
Prepare and configure IBM Cloud Operators...
namespace/ibmcloud-operator-configs created
configmap/ibmcloud-operator-config created
secret/ico-test-ibmcloud-operator-secret created
configmap/ico-test-ibmcloud-operator-defaults created
...Done.

The following listing shows the oc get commands and output to verify the create Secret and ConfigMap:

$ oc get secret -n ibmcloud-operator-configs
NAME                                TYPE                                  DATA   AGE
...
ico-test-ibmcloud-operator-secret   Opaque                                2      5d27s


$ oc get cm -n ibmcloud-operator-configs
NAME                                  DATA   AGE
ico-test-ibmcloud-operator-defaults   6      5d2h

You can use scripts/ico/install_ico.sh to install the operator, or you can do it manually through the OpenShift web console:

  1. In the OpenShift web console, click Operators in the left navigation menu, and then click OperatorHub.
  2. Search for IBM Cloud Operator (current version: 1.0.0), and click on the card for IBM Cloud Operator.
  3. Click Continue if you see a warning about showing the community operator.
  4. Click Install to perform a basic install with default values.

The IBM Cloud Operator is now ready to provision a backing service from the IBM Cloud catalog.

Provision and bind a service

A running IBM Cloud Operator can provision new and existing services and bind them to a namespace. You need the following information to provision a service:

  • The service class identifying the service from the IBM Cloud catalog
  • The plan to select the pricing model

If you want to use an already provisioned and existing service instance on your IBM Cloud Account, you can use the Alias plan. Alias indicates to the IBM Cloud Operator to use the existing instance with the service instance name instead of creating and provisioning a completely new instance from the IBM Cloud catalog.

To identify the service class, enter the following search command into the IBM Cloud CLI:

ibmcloud catalog search --type service --fields overview_ui.en.display_name,kind,name,id postgresql

You see the following search result for PostgreSQL:

ID                                     Display_name                                                              Kind      Name                               Id   
databases-for-enterprisedb-group       Databases for EnterpriseDB                                                service   databases-for-enterprisedb-group   databases-for-enterprisedb-group   
databases-for-postgresql-group         Databases for PostgreSQL                                                  service   databases-for-postgresql-group     databases-for-postgresql-group   
024d3260-5cf4-11e9-90a2-37a496589afc   Hyper Protect DBaaS for PostgreSQL                                        service   hyperp-dbaas-postgresql            24d3260-5cf4-11e9-90a2-37a496589afc   
oss.compose-for-postgresql             OSS Record: Compose for PostgreSQL(compose-for-postgresql)                oss       oss.compose-for-postgresql         oss.compose-for-postgresql   
oss.databases-for-postgresql           OSS Record: Databases for PostgreSQL(databases-for-postgresql)            oss       oss.databases-for-postgresql       oss.databases-for-postgresql   
oss.hyperp-dbaas-postgresql            OSS Record: Hyper Protect DBaaS for PostgreSQL(hyperp-dbaas-postgresql)   oss       oss.hyperp-dbaas-postgresql        oss.hyperp-dbaas-postgresql

You can also get details about a service using the value from the NAME column from the previous output. To see these details, including which plans exist, (for example, for hyperp-dbaas-postgresql), use the IBM Cloud CLI:

$ cloudshell:~$ ibmcloud catalog service hyperp-dbaas-postgresql

The following output displays the service details for Hyper Protect DBaaS for PostgreSQL with the different plans:

Getting catalog entry...
OK

ID                 024d3260-5cf4-11e9-90a2-37a496589afc   
Name               hyperp-dbaas-postgresql   
Kind               service   
Provider           IBM   
Tags               apidocs_enabled, data_management, ibm_created, rc_compatible, service_endpoint_supported   
Active             true   
Description        Deploy and manage a PostgreSQL cluster to protect your sensitive data in a Secure Service Container on IBM LinuxONE.   
Bindable           false   
Original Name         
RC Compatible      true   
RC Provisionable   true   
IAM Compatible     true   
Children           Name                                 Kind         ID                                                   Location   Original Location   Target      
                   postgresql-flexible                  plan         c8550ed3-894b-462d-98ee-68e80e3955d4                                                      
                   |__postgresql-flexible-au-syd-rc     deployment   c8550ed3-894b-462d-98ee-68e80e3955d4:au-syd47034     au-syd                         bluemix-au-syd      
                   |__postgresql-flexible-eu-de-rc      deployment   c8550ed3-894b-462d-98ee-68e80e3955d4:eu-de19369      eu-de                          bluemix-eu-de      
                   |__postgresql-flexible-us-south-rc   deployment   c8550ed3-894b-462d-98ee-68e80e3955d4:us-south79470   us-south                       bluemix-us-south      
                   postgresql-free                      plan         ab547763-605f-4d83-a52e-f646249c8f89                                                      
                   |__postgresql-free-au-syd-rc         deployment   ab547763-605f-4d83-a52e-f646249c8f89:au-syd47847     au-syd                         bluemix-au-syd      
                   |__postgresql-free-eu-de-rc          deployment   ab547763-605f-4d83-a52e-f646249c8f89:eu-de95742      eu-de                          bluemix-eu-de      
                   |__postgresql-free-us-south-rc       deployment   ab547763-605f-4d83-a52e-f646249c8f89:us-south41360   us-south                       bluemix-us-so

Gather the following information from the previous output, which the IBM Cloud Operator uses to request the PostgreSQL service from the IBM Cloud catalog:

  • Service name for the serviceClass attribute (here: hyperp-dbaas-postgresql)
  • Desired plan (here: postgresql-free)
  • Region (here: eu-de), especially if this does not meet the default configured region in the IBM Cloud Operator configuration

The YAML file (scripts/ico/svc.ico.yaml) contains service and binding definitions for an existing service (Language Translator) and a new service ([Hyper Protect PostgreSQL). Adjust the YAML file with your previously gathered information if you want and apply the file to request and bind the services:

$ cd scripts/ico
$ oc apply -f svc.ico.yaml

After the binding, you have the following new resources in the custom namespace (here: ico-test):

  • A new secret holding all connection information in your namespace:

      $ oc get secrets
      NAME                       TYPE                                  DATA   AGE
      ...
      language-translator-demo   Opaque                                6      7m15s
      ...
    
  • A service.ibmcloud.ibm.com, which represents the IBM Cloud service:

      $ oc get service.ibmcloud
      NAME                              STATUS   AGE
      language-translator-demo          Online   6m46s
      postgresql-hyperprotect-tracker   failed   6m46s
    
  • A binding.ibmcloud.ibm.com, which manages the bind:

      $ oc get binding.ibmcloud
      NAME                              STATUS   AGE
      language-translator-demo          Online   7m12s
      postgresql-hyperprotect-tracker   Failed   7m12s
    

You now have two services provisioned from the IBM Cloud catalog, and the connection information is provided in the cluster without manual interaction. The configuration and management for the (back-end) services can now also be part of the source code and the other deployment configurations (here: svc.ico.yaml). Seamless GitOps, here we come!

Verify logs in case of trouble

In case the binding of a service is not successful, verify the logs of the ibmcloud-operator, which handles the service provisioning and secret creation. For example, if a parameter is missing or the service ID does not have enough permissions, you will see the related error messages here:

$ oc logs -f -n openshift-operators -l control-plane=controller-manager -c manager
...
2020-12-21T18:07:01.301Z        INFO    controllers.Binding     ibmcloud/ibmcloud.go:83 Context {"binding": "ico-test/language-translator-demo", "is": {"region":"eu-de","resourcegroupid":"b223af36cdcc4c8eae5f020ef498bc0e","resourcelocation":"eu-de","user":"serviceid-ico"}}
2020-12-21T18:07:01.301Z        INFO    controllers.Binding     ibmcloud/ibmcloud.go:360        Found ibmcloud-operator-config in own namespace, using that namespace for configuration {"binding": "ico-test/language-translator-demo", "own namespace": "openshift-operators", "management namespace": "ibmcloud-operator-configs"}
2020-12-21T18:07:01.427Z        INFO    controllers.Binding     controllers/binding_controller.go:451   Creating        {"credentials": "language-translator-demo"}
2020-12-21T18:07:03.439Z        INFO    controllers.Binding     controllers/binding_controller.go:483   Creating        {"secret": "language-translator-demo"}
2020-1
...

Delete a service

Deleting a service is simple and follows the usual process in Kubernetes and OpenShift: Use the same YAML definitions with oc delete -f. This also deletes the service on IBM Cloud. If you use an existing service – with the Alias plan, only the internal configuration and secret is deleted.

Helpful information

Use the API documentation for the IBM Cloud catalog service to get a better understanding of which parameters are relevant and could be used in the parameters section for the service provisioning.

The API documentation is always linked in the catalog service detail page (for example, Catalog: Hyper Protect DBaaS for PostgreSQL) or in the reference section of the documentation (Docu: Hyper Protect DBaaS for PostgreSQL).

Additionally, here are a few helpful commands:

  • List policy details for a service ID: ibmcloud iam service-policies serviceid-ico
  • Update policy using service ID and policy ID: ibmcloud iam service-policy-update serviceid-ico 5be.... --roles Manager --resource-group-name Default
  • Get all provisioned instances: ibmcloud resource service-instances --long
  • Get details to a service: ibmcloud catalog service hyperp-dbaas-postgresql
  • Search the catalog: ibmcloud catalog search --type service --fields overview_ui.en.display_name,kind,name,id db2

Red Hat Service Binding Operator

The Red Hat Service Binding Operator provides a solution for binding credentials or other sensitive information from a backing service to an individual application.

Generally, the following activities are manual tasks and error-prone:

  • Retrieve the sensitive information (credentials, connection details, and more) from the backing service
  • Create a Secret or ConfigMap holding the retrieved information
  • Link them to an individual application

During these activities, the sensitive data is visible for the operator/administrator, which is not necessarily desirable (for example, in regulated environments).

The Service Binding Operator takes care of all the steps listed for the operating team, in an automated way. The new custom resource ServiceBinding defines the relationship between the backing service and the application. The Service Binding Operator gathers the relevant information from the backing service and creates a Secret or ConfigMap and injects it into the application.

To achieve this, the backing service (operator) must have corresponding annotations in order to expose the relevant information for a consuming application (via the Service Binding Operator).

The result is that you can define the intent to use a backing service, and the Service Binding Operator binds the service to the application. Everything is defined as a normal Kubernetes YAML file.

Prepare and install the Service Binding Operator

To install the operator, use the OpenShift web console and install Red Hat Service Binding Operator (0.4.0 provided by Red Hat), or use the following script scripts/rsb/install_rsb.sh to install the operator:

$ cd scripts/rsb
$ ./install_rsb.sh
Install Red Hat Service Binding Operator...
subscription.operators.coreos.com/rh-service-binding-operator created
...Done.

This installs the Red Hat Service Binding Operator, which observes new ServiceBinding resources.

Test the binding with the help of the Service Binding Operator

To test the service binding, you use the the Language Translator service that you provisioned from the previous section with the help of IBM Cloud Operator.

The following exemplary application uses the Language Translator service and expects two environment variables: SVC_LANGUAGE_TRANSLATOR_URL and SVC_LANGUAGE_TRANSLATOR_APIKEY.

The following commands deploys a test app:

$ oc new-app --name app-svc-binding-demo -i nodejs https://github.com/haf-tech/tekton-101 -e PORT=8080 -e TEKTON_101_ENV_EXAMPLE=Version-0.1
...

This command exposes the route:

$ oc expose svc app-svc-binding-demo
route.route.openshift.io/app-svc-binding-demo exposed

This command gets the created route and tests the endpoint:


$ oc get route app-svc-binding-demo --template='http://{{.spec.host}}'
http://app-svc-binding-demo-ico-test.apps.cluster-608e.sandbox1691.opentlc.com%

$ curl $(oc get route app-svc-binding-demo --template='http://{{.spec.host}}')/translate/en/Sicherheit
{"translations":[],"word_count":0,"character_count":0,"message":"No backend service defined"}%

The running instance of the test application is currently not bound to the backing service Language Translator. You see in the last output the response of the requested endpoint with the status message No backend service defined.

As soon as you apply the ServiceBinding definition, the Service Binding Operator processes this request, determines the credentials from the backing service, creates the secret, and restarts the application deployment again with the new Secret. The following is an excerpt of the ServiceBinding definition:

apiVersion: operators.coreos.com/v1alpha1
kind: ServiceBinding
metadata:
  name: svc-binding-lang-translate-nodejs
  namespace: ico-test
spec:
  services:
  - group: ibmcloud.ibm.com
    version: v1
    kind: Binding
    name: language-translator-demo
    id: tr
  application:
    name: app-svc-binding-demo
    group: apps
    version: v1
    resource: deployments
  customEnvVar:
     - name: SVC_LANGUAGE_TRANSLATOR_URL
       value: '{{ .tr.status.secretName.url }}'
     - name: SVC_LANGUAGE_TRANSLATOR_APIKEY
       value: '{{ .tr.status.secretName.apikey }}'

The services block holds the reference information to the Binding object for Language Translator, while the application block holds the reference to the application, which expects the secrets. The customEnvVar allows the possibility to map a field from the Secret to another environment variable.

Apply this service binding definition:

$ oc apply -f sb.lang-translator.yaml.yaml
servicebinding.operators.coreos.com/svc-binding-lang-translate-nodejs created

The Red Hat Service Binding Operator reacts on the created ServiceBinding, generates a new secret with the name svc-binding-lang-translate-nodejs, and fetches the details from the secret of the IBM Cloud Operator binding for the language-translator-demo service instance. It also restarts the deployment matching the name app-svc-binding-demo.

The following commands give some insight about the changes that have been made. oc get secrets displays the available Secrets, including the newly created one:

NAME                                TYPE                                  DATA   AGE
...
language-translator-demo            Opaque                                6      17m
svc-binding-lang-translate-nodejs   Opaque                                4      5m54s

To retrieve the content of the Secret holding the credentials for the Language Translator service, use the command oc get secrets svc-binding-lang-translate-nodejs -o yaml | grep '^data:' -A3:

data:
  SVC_LANGUAGE_TRANSLATOR_APIKEY: aFJRVF...
  SVC_LANGUAGE_TRANSLATOR_URL: aHR0cHM...

If you recall the endpoint of the application again, this time you get a positive response (curl $(oc get route app-svc-binding-demo --template='http://{{.spec.host}}')/translate/en/Sicherheit):

{"translations":[{"translation":"Security"}],"word_count":1,"character_count":10,"detected_language":"de","detected_language_confidence":0.9066952183967676}%

Even the test application now contains the environment variables with the API Key and URL to the service instance:

$ oc exec app-svc-binding-demo-745f698587-czqpq -- env | grep -i '^SVC'
SVC_LANGUAGE_TRANSLATOR_APIKEY=hRQTUaBt....
SVC_LANGUAGE_TRANSLATOR_URL=https://api.eu-de.language-translator.watson.cloud.ibm.com/instances/.....

You can verify that the test application can interact with Language Translator service without manual and direct credential assignment. The Red Hat Service Binding Operator has transparently taken over this assignment.

Summary

Infrastructure as Code, Configuration as Code, and GitOps are all terms circulating the industry right now as the trend is toward automating all manual activities on the pipeline and making all configuration part of the source code. Managing Secrets has always been complicated because storing them in the same source code repository is not a solution.

The solution presented in this tutorial solves the requirements and challenges in the sense of GitOps and helps on the journey toward higher automation and stable, secure configuration.