By Hendrik Van Run, Valentina Birsan | Published June 20, 2018
IBM Cloud Private is an application platform for developing and managing on-premises, containerized applications. The platform can be deployed off-premises on a public cloud provider or on-premises within a client data center. It is an integrated environment for managing containers that includes the Kubernetes container orchestrator, a private image repository, a management console, and monitoring frameworks.
The IBM Cloud Private pattern covered in this tutorial is a PureApplication system pattern that deploys an IBM Cloud Private application in different topologies, for a Community Edition (CE) or an Enterprise Edition (EE). The pattern offers a fast, reliable, and reproducible approach to installing and managing the IBM Cloud Private application on a PureApplication System.
This tutorial shows you how to install the IBM Cloud Private CE or EE on PureApplication environments using the IBM Cloud Private Pattern. It covers the steps required to install the pattern, describes the IBM Cloud Private topologies the pattern templates deploy, and the administrative operations available after deployment.
The IBM Cloud Private pattern is supported on both Intel and Power IBM PureApplication environments:
Note: When using a Power-based IBM PureApplication System W3700 environment, clients must bring their own Linux Power PC 64 Little Endian virtual image.
Patterns are available for both Intel and Power from IBM Fix Central as an IBM PureApplication emergency fix.
After you download the pattern, follow the instructions in IBMCloudPrivatePattern_QSG.pdf to install the pattern on your PureApplication environment. You need to have the pattern installed on your PureApplication environment before you can follow the steps described in this tutorial.
Both the Intel and Power pattern types have a dependency on the Docker pattern type version 220.127.116.11. This pattern type is included with PureApplication 2.2.5 on Intel. The pattern is also supported on PureApplication 2.2.3 and 2.2.4 Intel environment, but you will have to download and install the Docker pattern type version 18.104.22.168 or higher. This pattern type is available as part of the “Group_Content_PureApplicationSystem_22.214.171.124_Intel” on IBM Fix Central.
Note: On a Power-based IBM PureApplication System W3700 environment, you must download and install the Docker Pattern Type version 126.96.36.199 here on IBM Fix Central. (It is not part of “Group_Content_PureApplicationSystem_188.8.131.52_Power”.)
In addition to the IBM Cloud Private Pattern and its dependencies, you also need to ensure that you have the following in place when using Intel-based PureApplication environments:
Note: IBM PureApplication Software clients have to bring their own Red Hat Enterprise Linux 7.x virtual image.
The IBM Cloud Private pattern type includes a set of virtual system patterns that can be used to deploy an IBM Cloud Private cluster. These virtual system patterns are effectively a set of templates that can be found from the PureApplication user interface under Patterns > Virtual System Patterns. By simply entering “IBM Cloud Private” as a filter, only the IBM Cloud Private virtual system patterns will be shown.
These template virtual system patterns allow you to deploy a number of different IBM Cloud Private topologies. Refer to the IBM Cloud Private Knowledge Center to learn more about the architecture of IBM Cloud Private.
For templates 5 and 6 above, you must have a GPFS Shared Service instance deployed in the same environment profile with this deployment. Make sure the file system name you specify on the GPFS Client Policy Master node matches the file system name available on the GPFS Server instance that the GPFS Shared Service points to, and that the file system size is at least 60GB.
Note: GPFS is not currently used to provide a highly available persistent storage provider for IBM Cloud Private.
All virtual system patterns except the IBM Cloud Private Test Environment support adding or removing IBM Cloud Private worker, proxy, or management nodes after deployment. This is done through Manual Scaling Operations from the Virtual System Instance console. Table 1 provides an overview of the types of IBM Cloud Private nodes, the number that are supported, and whether they can be scaled after deployment.
Note: IBM Cloud Private (and Kubernetes) only supports an odd number of master nodes. This is in order to be able to ensure quorum. Refer to High Availability IBM Cloud Private clusters in the IBM Cloud Private Knowledge Center for more details.
Note: IBM Cloud Private supports one or more proxy nodes, but the current IBM Cloud Private pattern has a limitation: It only supports an odd number of proxy nodes at deployment time; however, proxy nodes can be added or removed afterward.
In this tutorial, we will use the IBM Cloud Private virtual system pattern template to demonstrate the automated deployment of a more complex IBM Cloud Private topology as shown in the following diagram.
Select Patterns > Virtual System Patterns and select the IBM Cloud Private Virtual System Pattern, then click on the Deploy icon to start a new deployment.
The following pattern attributes will be displayed for the new deployment:
All required values have been set so that you can always deploy directly without making any changes. However, in this example we will make some changes.
This is the type of the IBM Cloud Private binaries that are used by the deployment—“IBM Cloud Private-ee” for Enterprise Edition (EE) or “IBM Cloud Private-ce” for Community Edition (CE).
The CE binaries are packaged with the IBM Cloud Private pattern on IBM Fix Central. You can also import other IBM Cloud Private binaries, CE or EE, and use them with the pattern. Refer to the IBM Cloud Private pattern documentation in IBMCloudPrivatePattern_QSG.pdf to see how to upload these binaries to the Storehouse that will be used by your deployment.
If the VMs deployed on your PureApplication environment have access to the internet, you can skip the step that uploads the CE binaries to the Storehouse. You must set the Installation type to “IBM Cloud Private-ce” and specify the version of the IBM Cloud Private Community Edition to be downloaded from Docker Hub. At deployment time, the pattern will first attempt to get the binaries from the Storehouse, but when the binaries are not found there, it will download them from Docker Hub.
Remember that if you want to deploy more than one master and/or proxy host:
This is the version of the binaries that’s uploaded to the Storehouse. You can set a different version here if you have uploaded other IBM Cloud Private binaries to the Storehouse. You can have more than one set of IBM Cloud Private binaries uploaded to the Storehouse; this could be a mix of CE and EE editions or different versions. Each deployment will pick up the right edition and version based on the selected installation type and IBM Cloud Private version properties. For example, you can deploy an IBM Cloud Private 184.108.40.206 CE instance using the IBM Cloud Private Test Environment template and an IBM Cloud Private 220.127.116.11 EE instance with the IBM Cloud Private template.
This is the password for the IBM Cloud Private console admin user. The default value is “admin,” but we strongly recommend changing this at deployment time.
Note: Both the username and password of the admin user can be changed after deployment. Refer to Changing the cluster administrator access credentials in the IBM Cloud Private Knowledge Center for detailed instructions.
The default installation directory for IBM Cloud Private is /ibm-cloud-private.
This is the port used by Kubernetes. The default is 8888, but this conflicts with the maestro agent of PureApplication which runs on all VMs and uses the same port. Therefore, the default for this port for the IBM Cloud Private pattern is 8989.
This is the port that’s used by the PureApplication internal inlet application. You should update it only if the PureApplication has been changed to use a different port. The default is 8888.
An HA boot host is present in all virtual system patterns except the “IBM Cloud Private Test Environment.” The name of this VM can be confusing as the IBM Cloud Private boot node is not necessarily HA and there is just a single IBM Cloud Private boot node.
When deploying IBM Cloud Private with multiple master and/or proxy nodes—or when you intend to scale to multiple proxy nodes after deployment—you must specify additional pattern parameters at deployment time:
When these pattern parameters are provided, virtual IP addresses are automatically assigned to handle multiple IBM Cloud Private master and proxy nodes. Under the covers, the virtual IP manager assigns the Virtual IP for Cluster Master HA to the network interface of the active IBM Cloud Private master nodes (as specified by “Ethernet interface name for Master HA”). Should that IBM Cloud Private master node become temporarily unavailable, the virtual IP manager will re-assign the virtual IP address to one of the remaining IBM Cloud Private master nodes. So if you select to deploy multiple IBM Cloud Private master nodes, the Virtual IP for Cluster Master HA will also be used to access the IBM Cloud Private console.
The mechanism described here also applies to the Virtual IP for Proxy Master HA. Refer to High availability IBM Cloud Private clusters in the IBM Cloud Private Knowledge Center for more details.
Note: When using the Virtual IP for Cluster Master HA or Virtual IP for Proxy Master HA pattern parameters, make sure that those IP addresses are unique, available, and accessible from the corresponding network interface (i.e. eth1). One mechanism for doing this would be to deploy a dummy base OS virtual system instance, stop it, and then use the IP address of the VM of this instance as the virtual IP of the IBM Cloud Private pattern deployment.
As you saw in the topology diagram above, this tutorial uses the following deployment options for the various IBM Cloud Private nodes. There is always just a single HA boot node.
Note: All types of nodes except the master can still be scaled after deployment, as well. So adding an additional worker node to handle more deployments within the IBM Cloud Private cluster is not a problem. We will demonstrate this later.
Follow these steps to configure the deployment options for the IBM Cloud Private nodes:
When deployment has completed, examine your IBM Cloud Private Virtual System Instance. It should consist of a single boot node, three master nodes, one proxy node, two management nodes and two worker nodes. And of course a single IBM Cloud Private boot node.
After the Virtual System Instance has been deployed, you can access the IBM Cloud Private console.
You can use the IBM Cloud Private Virtual System Instance Console to add or remove IBM Cloud Private worker, proxy, or management nodes, extend the size of the disks used by the deployment, or view the IBM Cloud Private cluster status. The Instance Console can also be used to access log files to troubleshoot deployment errors.
As you saw earlier, you can specify the number of IBM Cloud Private worker nodes that are initially deployed. After deployment, you can add (scale out) or remove (scale in) those nodes.
Following a similar approach, you can add or remove IBM Cloud Private proxy and management nodes.
Use this operation on any of the IBM Cloud Private roles to extend the disk size for /var/lib/elasticsearch/data, /var/lib/docker, or /var/lib/registry. The IBM Cloud Private pattern sets the default disk size for these data paths as recommended by IBM Cloud Private. However, if your deployment requires it, you can use this operation to increase the disk size as needed.
[11/01/17 19:05:02 UTC] Current size for mount point /var/lib/docker is 59(G)
[11/01/17 19:05:02 UTC] Scale up mount point /var/lib/docker with extra 20(G)
[11/01/17 19:05:02 UTC] Begin to start scaleNode task for node Test_Environment_Host.11509542655979, disksize: 20
[11/01/17 19:05:59 UTC] current task 4435 is succeed
[11/01/17 19:05:59 UTC] End to check scaleNode task 4435, rc = 0
[11/01/17 19:05:59 UTC] Begin to handle post-scaleNode
[11/01/17 19:05:59 UTC] format new VMFS disk
[11/01/17 19:05:59 UTC] Begin to format new local disk
[11/01/17 19:05:59 UTC] End to format new disk, rc = 0
[11/01/17 19:05:59 UTC] succeeded in formatting local disk
[11/01/17 19:05:59 UTC] succeed to handle 3post scaleup on VM Test_Environment_Host.11509542655979
[11/01/17 19:05:59 UTC] End to handle post-scaleNode
[11/01/17 19:05:59 UTC] End to scaleNode task for Node Test_Environment_Host.11509542655979
[11/01/17 19:05:59 UTC] Current size for mount point /var/lib/docker is 79(G)
Sometimes things do not work as you would expect. The IBM Cloud Private pattern logs information in a number of different places. Let’s quickly review some of the most common ones.
The output of the operations described above can be accessed from the Logging View on each of the deployed virtual machines. On the deployment, click Manage > Logging then select any of the deployed virtual machine sections and expand IBMCloudPrivate /../ICp/logs.
The IBM Cloud Private product installation logs are available on the boot host only, located under IBMCloudPrivate /../ICp/logs/icp.log. The other hosts also log deployment information under the IBMCloudPrivate /../ICp/logs/icp.log.
The cluster hosts and config.yaml files are also available under the same IBMCloudPrivate folder.
Containers logs can be found in the IBMCloudPrivate../log/containers folder.
In this tutorial, you learned how to use the IBM Cloud Private virtual system pattern type to deploy and manage the IBM Cloud Private clusters in all topologies supported by the product. With this PureApplication pattern, you can install IBM Cloud Private in a fast, reliable, and reproducible manner. You can install different versions of the product (Community Edition or Enterprise Edition) directly from Docker Hub or without an internet connection.
Now you can get started working with IBM Cloud Private in your particular environment.
Acknowledgements: We would like to thank Joe Wigglesworth, Sandeep Minocha, and Dennis Lauwers for their help with this tutorial.
Think is IBM's flagship technology conference. This year, one of our leading containers dev advocates will be live streaming on…
February 6, 2019
January 28, 2019
Back to top