Digital Developer Conference: Hybrid Cloud 2021. On Sep 21, gain free hybrid cloud skills from experts and partners. Register now

IBM Cloud Pak for Multicloud Management Managed Services and Chef

IBM Cloud Pak for Multicloud Management Managed Services is shipped with a custom Terraform resource to orchestrate Chef operations. This component is called the CAMC Provider and invokes REST services exposed by the Pattern Manager, which in turn, configures a Chef server and initiates the Chef-client to run on target nodes. The middleware content library uses this provider to allow native Chef integration for the IBM-provided Terraform templates.

This article explores some of the considerations for Chef cookbook developers who want to use the CAMC Provider to extend the IBM-provided templates with their own cookbooks. The major principles explored are:

  • Role-based orchestration
  • IBM Cloud Pak for Multicloud Management Managed Services Parameter to Chef node mapping
  • Node identification and Chef search
  • Managing secrets


You must have some basic knowledge of Chef and the IBM Cloud Pak for Multicloud Management Managed Services Content Runtime. We recommend that you also understand the CAMC Provider.

Role-based orchestration

IBM-provided Terraform templates contain a custom CAMC resource that invokes the Pattern Manager to configure and run Chef-based resources on the target nodes. The basic unit of orchestration is the Chef role. This is an important distinction because the run list for each Chef node object is augmented by a series of Chef roles that are incrementally expanded as orchestration from Terraform runs. The order of orchestration is defined within the Terraform template through the depends_on parameter and the atomic invocations of each orchestration step invoked by the Terraform engine.

Resources might have an explicit execution order that provides the framework for executing roles in order across different virtual machines in the stack. Each subsequent invocation of the CAMC resource adds a new role to the Chef run list on the target node and execute the chef-client.

Role-based orchestration

The previous image highlights the role as the primary component that is subject to orchestration and the driver of the orchestration is Terraform, which can order (see the numbering) the execution of resources in an order defined within the template.

Granular roles

The implication for the design of cookbooks is to ensure that the Chef roles are sufficiently granular to uniquely describe an atomic action (for example, install, fixpack, or create-server). Finding the correct balance is essential because too many might result in overly complex patterns, and too few might result in insufficient flexibility to create patterns from the base cookbooks.

Idempotent cookbooks

An important side effect of an incrementally expanding Chef run list between chef-client executions is that Chef roles are run and rerun a number of times during the provisioning of a single template. A natural consequence of this is that all Chef recipes and resources must be idempotent, which means that there is no failure upon subsequent executions and that Chef created objects are be created multiple times.

Chef node object population

IBM Cloud Pak for Multicloud Management Managed Services supports a rich set of internal variables loosely known as Managed Services Variables (previously known as CAM Variables). Managed Services Variables is a metadata description of variables supported by IBM Cloud Pak for Multicloud Management Managed Services that can be passed into the template provisioning process. A notable field in Managed Services Variables is the default field, which describes a default value for a field.

It is important to understand how these variables are consumed by the CAMC Provider and the subsequent effect on the Chef node objects.

The following image shows the general form of the mapping.

CAMC Provider

Terraform templates contain different types of variables used for different purposes. The variables of interest to a Chef developer are the Middleware variables, which provide the multitude of variables required to install and configure middleware. Middleware variables are attached to various Terraform resources and in the case on CAMC resources, are transferred by the Pattern Manager to the Chef node object.

variable "wmq_qmgr_qmgr1_dlq" {
  type = "string"
  description = "Queue Manager dead letter queue"

In this example, the wmq_qmgr_qmgr1_dlq parameter might be entered by a user at deployment and the resultant variable populated in the Chef node object with a precedence level of default.

Parameter placement

When you understand how IBM Cloud Pak for Multicloud Management Managed Services passes deployment variables to the Chef node object, you can see where a parameter that is exposed becomes an important consideration. For example, default node attributes in Chef override default role attributes. The decision to expose a variable to IBM Cloud Pak for Multicloud Management Managed Services through Managed Services Variables has an implicit effect on the Chef attribute that preceded it. Therefore, deliberate decisions around a variable’s precedence level should be made early on. A common distinction is the decision to expose a variable in the Chef role or through a Node variable, remembering that IBM Cloud Pak for Multicloud Management Managed Services always populates the Node object for parameters that are passed to it through Terraform.

Complex multinode templates usually require software that is deployed on one node understanding aspects of software on another node. A typical example is a load balancer proxying 1..n web servers. In this case, the following might apply:

  • The web server needs a complete list of IP/PORT combinations of all downstream servers.
  • The list of downstream servers might not be known at deployment.
  • You would like to expand or reduce the set of downstream servers through dynamic scale-in and scale-out operations.

Your choices often get reduced to providing a list of downstream servers at deployment or providing a dynamic mechanism that determines the current list of downstream servers based on a tagged attribute. The problem then becomes one of server identification, or wanting to uniquely understand the function of a Chef node based on certain attributes.

IBM Cloud Pak for Multicloud Management Managed Services supports this notion by providing a Chef attribute to each deployed node that describes the function of the node.

Search Chef node

In the previous example, a Chef node might be searched based on the stack_name and roles attributes of the Chef node object to identify the function of the node. This method is preferable to attempting to hardcode concepts such as IP address in Managed Services Variables because this tends to complicate the deployment and promotes inflexible patterns that cannot respond to day2 changes such as adding or removing nodes from a pattern.

Dynamic targeting

Designers of Chef cookbooks that interact with software on other nodes should consider using Chef search and the stack_name and roles attributes to search for nodes of a specific type rather than relying on values in fixed variables. Chef nodes created by the CAMC resources automatically have the stack_name and roles attribute populated during template creation.

In the previous web server example, instead of interpreting a list of IP/Port combinations from a Chef attribute, it might be preferable to use Chef search to find a list of web servers based on the stack_name and roles attributes.

Managing secrets

Managing secrets in Chef can be effectively handled using the Chef Vault, which is a convenient wrapper around encrypted databags. The Chef Vault gem is included with each managed deployment of the content runtime.

The challenges for integrating into IBM Cloud Pak for Multicloud Management Managed Services are:

  • How to define a parameter as a secret
  • How to securely pass the parameter from Terraform to the Chef Vault
  • How can Chef recipes easily find and reference the stored secret

To address these broad challenges, support for Chef Vault has been built into the CAMC provider and into the Managed Services Variables construct. A secret in Managed Services Variables is defined using the secret field as follows:

    "name": "WASDMGRNode01_was_profiles_dmgr_keystorepassword",
    "type": "string",
    "description": "Specifies the password to use on keystore created during profile creation",
    "hidden": false,
    "label": "WebSphere default keystore password - WASDMGRNode01",
    "secured": true,
    "required": true,
    "immutable": false,
    "group_name": "wasdmgrnode01"

Note that the secret field is set to true.

During the deployment, the parameters pass from Terraform through the Pattern Manager and into Chef in the following manner.

Passing parameters from Terraform

Terraform resources need the following basic attributes to ensure that a new Chef Vault is created for each stack and that secret parameters are properly populated. For example, the following code is a snippet from the camc_bootstrap resource.

"ibm_internal": {
  "stack_id": "${random_id.stack_id.hex}",
  "stack_name": "${var.ibm_stack_name}",
  "vault": {
       "item": "secrets",
       "name": "${random_id.stack_id.hex}"

The camc_bootstrap resource is generally the first CAMC resource to be run. At this point, a minimal set of Chef node attributes is populated and the Chef Vault is created for the stack. Additional middleware variables are defined in the camc_softwaredeploy resource. For example, the following code is a snippet from the camc_softwaredeploy resource.

"vault_content": {
  "item": "secrets",
     "values": {"was": {"profiles": {"dmgr": {"keystorepassword": "${var.WASDMGRNode01_was_profiles_dmgr_keystorepassword}"}}}},
     "vault": "${random_id.stack_id.hex}"

Handling secrets

When consuming secrets from a Chef cookbook perspective, the following constructs are provided by the CAMC provider and the Pattern Manager to create a unique encrypted database for each stack deployed.

  • The encrypted database created is referred to by the ibm_internal:name node variable.
  • The item within this database to use is referenced by the ibm_internal:item node variable.
  • A structure of all secrets is stored in the vault under the name of values.

Ensure that these fields are referenced in the cookbook to ensure that the cookbook can be correctly driven by a CAM template deployment.


This article explained how to use the custom Terraform resource CAMC Provider with Chef cookbooks. After reading this article, you have a better understanding of how to use the CAMC Provider to integrate your Chef cookbooks with IBM Cloud Pak for Multicloud Management Managed Service.

Next steps

We encourage you to review Chef guidelines with Managed Services for more information regarding running Chef on IBM Cloud Pak for Multicloud Management Managed Service. A large set of predefined templates that use CAMC Provider exists in GitHub to get you started. You can use these as examples to create your own set of templates to integrate your Chef cookbooks with IBM Cloud Pak for Multicloud Management Managed Service.