Step-by-step
-
Create a Kubernetes Cluster on IBM Cloud via CLI
Kubernates Cluster can be creater either via CLI or using IBM Cloud Web Interface. In this receipe we will be following CLI methology.
- Login to IBM Cloud, using initenv.sh (Refer Appendix for more info)
(modify initenv.sh for federated user accounts on IBM Cloud, replace bx login -a with bx login -sso)
Sample:if [[ "${INTERACTIVE_MODE}" = "true" ]]then
bx login -sso$ ./initenv.sh
Properties file (./args/initenv.properties) can also be used to initialize & login to IBM Cloud.
- If you have multiple IBM Cloud accounts, select the account where you want to create your Kubernetes cluster.
- If you want to create or access Kubernetes clusters in a region other than the IBM Cloud region that you selected earlier, run
$ bx cs region-set - Review the locations that are available. The locations that are shown depend on the IBM Cloud Container Service region that you are logged in.
$ bx cs locations(Selecting AP South- syd01 location for this demo)
- Choose a location and review the machine types available in that location. The machine type specifies the virtual compute resources that are available to each worker node.
$ bx cs machine-types <location> - Check to see if a public and private VLAN already exists in the IBM Cloud infrastructure (SoftLayer) for this account.
- Run the cluster-create command.
$ bx cs cluster-create –location syd01 –machine-type u2c.2×4 –hardware <shared_or_dedicated> –public-vlan <public_vlan_id> –private-vlan <private_vlan_id> –workers 3 –name <cluster_name> –kube-version <major.minor.patch> - Verify that the creation of the cluster was requested.
$ bx cs clustersNote: It can take up to 15 minutes for the worker node machines to be ordered, and for the cluster to be set up and provisioned in your account.
When the provisioning of your cluster is completed, the status of your cluster changes to deployed. -
Check the status of the worker nodes.
$ bx cs workers <cluster>When the worker nodes are ready, the state changes to normal and the status is Ready. When the node status is Ready, you can then access the cluster.
Note: Every worker node is assigned a unique worker node ID and domain name that must not be changed manually after the cluster is created. Changing the ID or domain name prevents the Kubernetes master from managing your cluster.
-
Set the cluster you created as the context for this session. Complete these configuration steps every time that you work with your cluster. Get the command to set the environment variable and download the Kubernetes configuration files.
$ bx cs cluster-config <cluster_name_or_id>When the download for the configuration files is finished, a command is displayed that you can use to set the path to the local Kubernetes configuration file as an environment variable.
-
Copy and paste the command that is displayed in your terminal to set the KUBECONFIG environment variable.
-
Verify that the KUBECONFIG environment variable is set properly.
$ echo $KUBECONFIG -
Launch your Kubernetes dashboard with the default port 8001.
$ kubectl proxy -
Open the below URL in a web browser to see the Kubernetes dashboard.
http://localhost:8001/ui
- Login to IBM Cloud, using initenv.sh (Refer Appendix for more info)
-
Create Database Service (DB2) on IBM Cloud
DB2 is used to create the tables required for Mobile First Platform & store product specific data.
- Navigate to https://console.bluemix.net/
- Use user login credentials & select the desired Account, Region & space.
- Click Create Resource button available on Dashboard
- Filter the Catalog for DB2
- Click on DB2, Enter the Service Name & Click Create
- From the list of services created, Click Service Credentials from Left menu
- Add & Save new Service credentials from +New Credentials button available on screen
- Created credentials can be accessed from View Credentials options
-
Instantiate Namespace for IBM Container Service on IBM Cloud
You need to create your own private namespace in the container registry. The namespace is a unique name to identify your private repository on the IBM Cloud registry. Choose a namespace according to following rules:
- It can contain only lowercase letters, numbers, or underscores (_).
- It can be 4 – 30 characters. If you plan to manage containers from the command line, you might prefer to have a short namespace that can be typed quickly.
- It must be unique in the IBM Cloud registry.
Namespace is appended to the registry URL as follows: registry.<region>.bluemix.net/my_namespace
For more information on IBM Cloud Registry URLs, refer Appendix
Follow below instructions to instantiate a namespace.-
Log in to IBM Cloud, as done in step 1.1
-
Add a namespace to create your own image repository
$ bx cr namespace-add mfpv8namespace -
To ensure that your namespace is created, use below command to get list of all the namespaces
$ bx cr namespace-list
-
Create Docker Image and Push to IBM Cloud Registry
-
Build the MobileFirst server image and upload to IBM Container repository
$ ./prepareserver.sh – The script will build the MFP server image with the customizations done to ‘usr-mfpf-server’ and pushes the images to the IBM Containers Service.
Note: To use specific namespace, use ./args/prepareserver.properties as cmd param for prepareserver.sh after modifying SERVER_IMAGE_TA
(refer Appendix for parameter specification) -
After successful push, check the MFP image list:
$ bx cr image-list
-
-
Install IBM MFP 8.0 on Kubernetes Cluster
-
Configure the databases for MobileFirst Platform
$ ./prepareserverdbs.sh – This script configures the MFPF server with the database selected. The tables which are required by MobileFirst Platform are also created.
Use ./args/prepareserverdbs.properties as cmd param for prepareserverdbs.sh after entering values for DB Properties according to db2 service created in IBM Cloud. -
Set your terminal context to your cluster
$ bx cs cluster-config <my-cluster>In the output, the path to your configuration file is displayed as a command to set an environment variable, for example:
export KUBECONFIG=/Users/ibm/.bluemix/plugins/container-service/clusters/my-cluster/kube-config-prod-dal12-my-cluster.yml -
Create a Persistent Volume Claim.
$ kubectl create -f ./args/mfpf-persistent-volume-claim.yaml
(Review the yaml file args/mfpf-persistent-volume-claim.yaml and then run the command) -
Create the Kubernetes Deployments
$ kubectl create -f ./args/mfpf-deployment-server.yaml -
After creation, to use the Kubernetes UI, execute the following command
$ kubectl proxy -
Navigate to http://localhost:8001/ui in browser
-
-
Access MFPServer after installation
-
Access Cluster
Login in to IBM Cloud Console https://console.bluemix.net/dashboardUnder Clusters, click on the Cluster Created for MFP Installation
-
Get complete information regarding Cluster
Copy the Ingress subdomain URL from the given info. -
Access MFPConsole
Append /mfpconsole to the Ingress subdomain URL
E.g. http://<cluster-name>.ap-south.containers.mybluemix.net/mfpconsoleCopy the URL in browser, & navigate to URL
Login Credentials for MFPConsole can be configured in mfp-deployment-server.yaml file.
-
-
Enable SSL
To enable SSL, update the Ingress on Kubernetes Cluster. An Ingress is a collection of rules that allow inbound connections to reach the cluster services.
Snippet of Ingress yaml file for SSL enablement:
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: mfp-ingress
namespace: default
annotations:
ingress.bluemix.net/client-max-body-size: "200m"
ingress.bluemix.net/proxy-read-timeout: "300s"
ingress.bluemix.net/proxy-connect-timeout: "300s"
ingress.bluemix.net/redirect-to-https: "true"
spec:
tls:
- hosts:
- **************.mybluemix.net
secretName: mfpprodcluster
rules:
- host: ********.mybluemix.net
http:
paths:
- path: /mfpconsole
backend:
serviceName: mfpserver
servicePort: 9080
- path: /mfp
backend:
serviceName: mfpserver
servicePort: 9080
- path: /imfpush
backend:
serviceName: mfpserver
servicePort: 9080
- path: /mfpadmin
backend:
serviceName: mfpserver
servicePort: 9080
- path: /mfpadminconfig
backend:
serviceName: mfpserver
servicePort: 9080
- path: /doc
backend:
serviceName: mfpserver
servicePort: 9080Re-deploy the server, referring latest docker image using kubectl
$ kubectl create -f <new-ingress>.yaml -
Enable Auto scaling
With Horizontal Pod Auto scaling, Kubernetes automatically scales the number of pods in a replication controller, deployment or replica set based on observed CPU utilisation.
- Access the kubernetes dashboard, follow the steps 1.11 to 1.15
- Navigate to Pods, from left menu
- Select the pods, one-by-one & click Edit
- Copy the configuration JSON & modify object/spec/containers[0]/resources
Add below object,"requests": {
"cpu": "500m"
}Final Object object/spec/containers[0]/resources:
"resources": {
"limits": {
"cpu": "1"
},
"requests": {
"cpu": "500m"
}
} - Delete the pod & Create the pod using “+Create” option available on dashboard
- Upload the modified JSON
- Repeat Step 4-6, for all the pods
- Get the kubectl deployments
$ kubectl get deployment - Set the autoscale
$ kubectl autoscale deployment <deployment-name> –cpu-percent=60 –min=2 –max=5 - Get HPA for the deployment
$ kubectl get hpa - Access the kubernetes dashboard, & confirm Horizontal Pod Autoscalers
-
Enable logs
To enable logs, execute below commands.
- $ bx cs logging-config-create <cluster-name> –logsource application –type ibm –hostname ingest-au-syd.logging.bluemix.net –port 9091 –org <organization-name> –space <space>
- $ kubectl exec <mfp-servername> -i -t — bash -il
-
Manage User Roles for MFP
To add additional users & specify the roles according to the groups, follow below steps.
- List down all the required users & passwords in ./args/mfpf-deployment-server.yaml, based on the requirement.
Under ConfigMap – properties. Sample config-maps.yaml:apiVersion: v1
kind: ConfigMap
metadata:
name: mfp-configmap-server
namespace: default
data:
mfpf.properties: |-
MFPF_ADMIN_USER=admin
MFPF_ADMIN_PASSWORD=Password@1234
ANALYTICS_CLUSTER_NAME=mfpanalytics
INGRESS_HOST_FOR_ANALYTICS_CONSOLE=########.mybluemix.net
ANALYTICS_ADMIN_USER=admin
ANALYTICS_ADMIN_PASSWORD=admin
MFPF_ADMIN_DB2_SERVER_NAME=######.bluemix.net
MFPF_ADMIN_DB2_PORT=50000
MFPF_ADMIN_DB2_DATABASE_NAME=BLUDB
MFPF_ADMIN_DB2_USERNAME=bluadmin
MFPF_ADMIN_DB2_PASSWORD=#######
MFPF_ADMIN_DB2_SCHEMA=MFPDATA
SSL_CONNECTION=false
MFPF_RUNTIME_DB2_SERVER_NAME=######.bluemix.net
MFPF_RUNTIME_DB2_PORT=50000
MFPF_RUNTIME_DB2_DATABASE_NAME=BLUDB
MFPF_RUNTIME_DB2_USERNAME=bluadmin
MFPF_RUNTIME_DB2_PASSWORD=#######
MFPF_RUNTIME_DB2_SCHEMA=MFPDATA
MFPF_PUSH_DB2_SERVER_NAME=<Will be filled by prepareserverdbs.sh>
MFPF_PUSH_DB2_PORT=<Will be filled by prepareserverdbs.sh>
MFPF_PUSH_DB2_DATABASE_NAME=<Will be filled by prepareserverdbs.sh>
MFPF_PUSH_DB2_USERNAME=<Will be filled by prepareserverdbs.sh>
MFPF_PUSH_DB2_PASSWORD=<Will be filled by prepareserverdbs.sh>
MFPF_PUSH_DB2_SCHEMA=<Will be filled by prepareserverdbs.sh>
MFPF_ADMIN_USER1=USER-AD-01
MFPF_ADMIN_PASSWORD1=USER@01
MFPF_ADMIN_USER2=USER-AD-02
MFPF_ADMIN_PASSWORD2=USER@02
MFPF_DEV_USER1=USER-DE-01
MFPF_DEV_PASSWORD1=USER@03
MFPF_DEV_USER2=USER-DE-02
MFPF_DEV_PASSWORD2=USER@04
MFPF_RO_USER1=USER-RE-01
MFPF_RO_PASSWORD1=USER@05Added 2 admin users, 2 deployer & 1 read-only
- Update the user refernce in configuration file, <archive-root>/bmx-kubernetes/usr-mfpf-server/config/registry.xml
Sample file:
<?xml version="1.0" encoding="UTF-8"?>
<!-- Licensed Materials - Property of IBM 5725-I43 (C) Copyright IBM Corp.
2015, 2015. All Rights Reserved. US Government Users Restricted Rights -
Use, duplication or disclosure restricted by GSA ADP Schedule Contract with
IBM Corp. -->
<server>
<!-- You can overwrite this file in your docker container either to extend
the basic registry, or to replace the basic registry by an ldap registry -->
<basicRegistry id="mobilefirst" realm="mobilefirstRealm">
<!-- User for JMX operations - Used by the MFP Server -->
<user name="${env.MFPF_SERVER_ADMIN_USER}" password="${env.MFPF_SERVER_ADMIN_PASSWORD}" />
<user name="${env.MFPF_ADMIN_USER}" password="${env.MFPF_ADMIN_PASSWORD}" />
<user name="demo" password="${env.MFPF_ADMIN_PASSWORD}" />
<user name="${env.MFPF_ADMIN_USER1}" password="${env.MFPF_ADMIN_PASSWORD1}" />
<user name="${env.MFPF_ADMIN_USER2}" password="${env.MFPF_ADMIN_PASSWORD2}" />
<user name="${env.MFPF_DEV_USER1}" password="${env.MFPF_DEV_PASSWORD1}" />
<user name="${env.MFPF_DEV_USER2}" password="${env.MFPF_DEV_PASSWORD2}" />
<user name="${env.MFPF_RO_USER1}" password="${env.MFPF_RO_PASSWORD1}" />
<group name="${env.MFPF_ADMIN_GROUP}">
<member name="${env.MFPF_ADMIN_USER}" />
<member name="${env.MFPF_SERVER_ADMIN_USER}" />
<member name="${env.MFPF_ADMIN_USER1}" />
<member name="${env.MFPF_ADMIN_USER2}" />
</group>
<group name="${env.MFPF_DEPLOYER_GROUP}">
<member name="demo" />
<member name="${env.MFPF_DEV_USER1}" />
<member name="${env.MFPF_DEV_USER2}" />
</group>
<group name="${env.MFPF_MONITOR_GROUP}">
<member name="demo" />
<member name="${env.MFPF_RO_USER1}" />
</group>
<group name="${env.MFPF_OPERATOR_GROUP}">
<member name="demo" />
<member name="${env.MFPF_RO_USER1}" />
</group>
</basicRegistry>
</server> - Re-create the docker image & push to registry. Execute Step 4
- Re-deploy the server, referring latest docker image using kubectl
$ kubectl apply -f <new-config>.yamlSample yaml file:
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: mfpserver
namespace: default
spec:
replicas: 2
template:
metadata:
labels:
app: mfpserver
name: mfpserver
spec:
containers:
-
image: "********.bluemix.net/<namespace>/<image-name>:v1"
imagePullPolicy: Always
name: mfpserver
volumeMounts:
-
mountPath: /opt/ibm/MobileFirst/envs/mfpenvfile
name: mfpconfigvolume
-
mountPath: /opt/ibm/wlp/usr/servers/mfp/logs
name: mfpservervolume
subPath: serverlogs
imagePullSecrets:
-
name: bluemix-default-secret
volumes:
-
name: mfpservervolume
persistentVolumeClaim:
claimName: mfppvc
-
configMap:
name: mfp-configmap-server
name: mfpconfigvolume - Re-create config maps using config-maps.yaml, created in Step 1
- Verify the installation, Follow Step 6
- List down all the required users & passwords in ./args/mfpf-deployment-server.yaml, based on the requirement.
-
Appendix or Additional Information
- Permissions for Creating & Managing IBM Clusters
To Manage resources on IBM Cloud Containers, user should have access permissions for IBM Cloud Container Service.
Users / Manage User / Choose Access Type - IBM Bluemix Regions
A region is a geographic area that is accessed by a dedicated endpoint. IBM Cloud Container Registry registries are available in the following regions.
IBM Cloud Region IBM Cloud Registory URL AP South registry.au-syd.bluemix.net EU Central registry.eu-de.bluemix.net UK South registry.eu-gb.bluemix.net US South registry.ng.bluemix.net Above registry service provide infrastructure to store Docker images that can be accessed using the mentioned registry Host URL.
- Scripts & args files
Extract the downloaded archive file. Navigate to /bmx-kubernetes/scripts directory.
scripts/ contain following files & a sub directory args/
SCRIPT PURPOSE initenv.sh The script logs into the container service. The script is a prerequisite to run any of the following scripts. prepareappcenter.sh The script will build the Application Center image with the customizations done to ‘usr-mfp-appcenter’ and pushes the images to the IBM Containers Service. prepareappcenterdbs.sh This script configures the Application Center with the database selected. The tables which are required by Application Center are also created. prepareserver.sh The script will build the MFP server and analytics images with the customizations done to ‘usr-mfpf-server’ and ‘usr-mfpf-analytics’ and pushes the images to the IBM Containers Service. prepareserverdbs.sh This script configures the MFPF server with the database selected. The tables which are required by MobileFirst Platform are also created. readme.txt Readme file with detailed steps to be executed for installation of MFP Server & Application Center on Kubernetes Cluster Navigate to /bmx-kubernetes/scripts/args directory.
args/ contains following argument properties files with parameters to be used for commands & .yaml files required to run on kubernetes cluster for configuring mfp server & mfp application center.initenv.properties – initenv.properties file contains below properties required to run initenv.sh. Initenv.properties file can be passed to initenv.sh as argument. Script will log in & initialize the IBM Bluemix cmd environment with the property values passed.
PROPERTY VALUE BLUEMIX_API_URL Bluemix API endpoint URL.
Default value – https://api.ng.bluemix.netBLUEMIX_USER Bluemix User id or E-mail
E.g. abc@in.ibm.comBLUEMIX_PASSWORD Bluemix password BLUEMIX_ACCOUNT_ID Bluemix Account ID
E.g. a3b3b333d19e8a87d0fa7cc999999999BLUEMIX_ORG Organization name in IBM Bluemix BLUEMIX_SPACE Space name in IBM Bluemix prepareappcenter.properties – prepareappcenter.properties file contains below property required to run prepareappcenter.sh
SERVER_IMAGE_TAG – The name of the MFP Application Center image. It should be of the form <BluemixRegistry>/<PrivateNamespace>/<ImageName>:<Tag>prepareappcenterdbs.properties – prepareappcenterdbs.properties file contains below properties required to run prepareappcenterdbs.sh
PROPERTY VALUE DB_TYPE Database or Service for storing MobileFirst data.
E.g. dashDBDB2_HOST Hostname or IP where database service is hosted & configured.
E.g. *********.au-syd.bluemix.netDB2_DATABASE Database Name
E.g. BLUDBDB2_PORT Database Port
E.g. 50000DB2_USERNAME Database username to access the database
E.g. bluadminDB2_PASSWORD Password for the username to success the database APPCENTER_DB_SRV_NAME Database Service instance name for storing application center data APPCENTER_SCHEMA_NAME Database schema name for application center prepareserver.properties – prepareserver.properties file contains below property required to run prepareserver.sh
SERVER_IMAGE_TAG – The name of the MFP Server image. It should be of the form <BluemixRegistry>/<PrivateNamespace>/<ImageName>:<Tag>
ANALYTICS_IMAGE_TAG– The name of the MFP Analytics image. It should be of the form <BluemixRegistry>/<PrivateNamespace>/<ImageName>:<Tag>prepareserverdbs.properties – prepareserverdbs.properties file contains below properties required to run prepareserverdbs.sh
PROPERTY VALUE DB_TYPE Database or Service for storing MobileFirst data.
E.g. dashDBDB2_HOST Hostname or IP where database service is hosted & configured.
E.g. *********.au-syd.bluemix.netDB2_DATABASE Database Name
E.g. BLUDBDB2_PORT Database Port
E.g. 50000DB2_USERNAME Database username to access the database
E.g. bluadminDB2_PASSWORD Password for the username to success the database ADMIN_DB_SRV_NAME Database Service instance name for storing admin service data RUNTIME_DB_SRV_NAME Database Service instance name for storing runtime data RUNTIME_SCHEMA_NAME Database schema name ENABLE_PUSH Create database for push service. Accepted values are Y or N PUSH_DB_SRV_NAME Database Service instance name for push data PUSH_SCHEMA_NAME Database schema name for push service mfpf-deployment-server.yaml – mfpf-deployment-server.yaml is used to deploy the mfpf server to Kubernetes Cluster. Edit the yaml file, add fill the details. This file contains following important information:
PROPERTY VALUE MFPF_ADMIN_USER Admin Username for MFP Console MFPF_ADMIN_PASSWORD Admin password for MFP Console user ANALYTICS_CLUSTER_NAME Analytics Server Cluster Name ANALYTICS_ADMIN_USER Admin Username for MFP Analytics Console ANALYTICS_ADMIN_PASSWORD Admin password for MFP Analytics Console user Addition to above information, following details can also be accessed:
· Service Port Numbers
· Persistent Volume Information
· Ingress Information
· Container Images Information
· Hostname related Information
· MFP Server instances related Information
· All REST Endpoints
· configMap to make the environment variables available in MFP Server
- Permissions for Creating & Managing IBM Clusters