Overview

Skill Level: Any Skill Level

Linux command knowledge

Learn how to install IBM Cloud Private-ce 2.1.0, a Kubernetes-based platform, on RHEL 7.4

Ingredients

The infrastructure for your Kubernetes cluster must be created prior to following this guide. At minimum a master and single worker node are required. The machine specs used for testing are below. A minimum of 151 GB of disk space on the master node is recommended.

Master Node - 4 CPUs, 8 GB RAM, 80 GB disk, public/accessable IP
Worker Node(s) - 2 CPUs, 4 GB RAM, 40 GB disk

Step-by-step

  1. Review the System Requirements

    The system requirements doc can be found here. Note the hardware requirements when setting up your machines. All of the machines created must have network access to one-another.

     

  2. Architecture

    Familiarize yourself with the architecture of IBM Cloud Private found here. For this install a single machine will act as the master, boot and proxy node. Each worker node will have its own machine. The management node will not be used.

  3. Clone the Install Repo

    Clone the GitHub Repo https://github.com/jcantosz/icp-rhel-install to your master node.

    The repo contains a scripts folder with a series of of numbered scripts. Change (cd) to icp-rhel-install/scripts for the rest of the remaining steps.

    [cloudusr@jcantosz-rhel-master ~]$ git clone https://github.com/jcantosz/icp-rhel-install.git
    Cloning into 'icp-rhel-install'...
    remote: Counting objects: 36, done.
    remote: Compressing objects: 100% (33/33), done.
    remote: Total 36 (delta 10), reused 20 (delta 3), pack-reused 0
    Unpacking objects: 100% (36/36), done.

    [cloudusr@jcantosz-rhel-master ~]$ cd icp-rhel-install/scripts/

    [cloudusr@jcantosz-rhel-master scripts]$ ls
    00-variables.sh           01-update-hosts.sh      04-configure-os.sh     07-configure-installer.sh
    01-1-passwordless-ssh.sh  02-ssh-setup.sh         05-firewall-config.sh  08-install.sh
    01-2-bind-mounts.sh       03-install-packages.sh  06-get-installer.sh    09-kubeconfig.sh
  4. Fill in the Variables

    Using your favorite editor fill in the 00-variables.sh file.

    • SSH_KEY¬†– the path to¬†the private key file used to SSH into each of the workers.
    • SSH_USER – the user to SSH into the workers as.
    • PUBLIC_IP – The IP on which you plan to access the web¬†dashboard. This may be the same as the MASTER_IP if there is only one interface¬†for accessing the master
    • MASTER_IP – The (private) IP address of the master node
    • WORKER_IPS – An array of IP addresses, one per worker
    • WORKER_HOSTNAMES –¬†An array of hostnames, one per worker, the size and order must match WORKER_IPS

    Note that this file also contains links to the ICP install docs, the ICP wiki, and a wiki page with directions for joining the #ibm-cloud-private private channel on Slack

     

    [cloudusr@jcantosz-rhel-master scripts]$ cat 00-variables.sh 

    #!/bin/bash
    # ----------------------------------------------------------------------------------------------\\
    # Description:
    #   A basic installer for IBM Cloud Private-CE 1.2.0 on RHEL 7.4
    # ----------------------------------------------------------------------------------------------\\
    # Note:
    #   This assumes all VMs were provisioned to be accessable with the same SSH key
    #   All scripts should be run from the master node
    # ----------------------------------------------------------------------------------------------\\
    # System Requirements:
    #   Tested against RHEL 7.4 (OpenStack - KVM-RHE7.4-Srv-x64)
    #   Master Node - 4 CPUs, 8 GB RAM, 80 GB disk, public IP
    #   Worker Node - 2 CPUs, 4 GB RAM, 40 GB disk
    #   Requires sudo access
    # ----------------------------------------------------------------------------------------------\\
    # Docs:
    #   Installation Steps From:
    #    - https://www.ibm.com/support/knowledgecenter/SSBS6K_2.1.0/installing/prep_cluster.html
    #    - https://www.ibm.com/support/knowledgecenter/SSBS6K_2.1.0/installing/install_containers_CE.html
    #
    #   Wiki:
    #    - https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/W1559b1be149d_43b0_881e_9783f38faaff
    #    - https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/W1559b1be149d_43b0_881e_9783f38faaff/page/Connect
    # ----------------------------------------------------------------------------------------------\\

    export SSH_KEY=~/.ssh/id_rsa
    export SSH_USER=cloudusr

    export PUBLIC_IP=9.x.x.x
    export MASTER_IP=10.10.25.157

    # WORKER_IPS[0] should be the same worker at WORKER_HOSTNAMES[0]export WORKER_IPS=("10.10.25.158" "10.10.25.159")
    export WORKER_HOSTNAMES=("jcantosz-rhel-worker-1" "jcantosz-rhel-worker-2")

    if [[ "${#WORKER_IPS[@]}" != "${#WORKER_HOSTNAMES[@]}" ]]; then
      echo "ERROR: Ensure that the arrays WORKER_IPS and WORKER_HOSTNAMES are of the same length"
      return 1
    fi

    export NUM_WORKERS=${#WORKER_IPS[@]}
  5. Set Up Password-less SSH if needed

    If SSH communication using keyfiles is not already set up between the master and worker nodes, the script 01-1-passwordless-ssh.sh can be run to help accomplish this.

  6. Bind Mounted Volumes if needed

    If you have volume mounts you wish to use for installing ICP, the script 01-2-bind-mounts.sh can help you accomplish this. Identify which mounted volumes you wish to bind to which folder locations and fill in the MOUNT_DIR variable as appropriate.

    This is helpful when the base machine’s storage is insufficient for installing ICP and volume mounts are used.

  7. Run the Scripts

    Run each script 01-update-hosts.sh through 09-kubeconfig.sh in numerical order. Reviewing each script prior to running it.

    Each script has a section that runs on the master and a for loop that cycles through each of the worker’s hostnames¬†and runs the same or similar commands on the workers.

    Note the URL and credentials of the web UI after running script 08-install.sh

    [cloudusr@j scripts]$ ./08-install.sh
    ...
    PLAY RECAP **********************************************************************************
    10.10.25.157               : ok=200  changed=65   unreachable=0    failed=0   
    10.10.25.158               : ok=118  changed=48   unreachable=0    failed=0   
    10.10.25.159               : ok=118  changed=48   unreachable=0    failed=0   
    localhost                  : ok=215  changed=113  unreachable=0    failed=0   

    POST DEPLOY MESSAGE *************************************************************************
    UI URL is https://9.x.x.x:8443 , default username/password is admin/admin

    Playbook run took 0 days, 0 hours, 21 minutes, 3 seconds
  8. Test the kubectl Commands

    After script 09-kubeconfig.sh has completed, your master node should have a working version of kubectl. Try running kubectl get pods –all-namespaces to see what is running on Kubernetes

    [cloudusr@j scripts]$ ./09-kubeconfig.sh 
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100 49.9M  100 49.9M    0     0  3843k      0  0:00:13  0:00:13 --:--:-- 5989k
    Cluster "cfc-cluster" set.
    Context "kubectl" created.
    User "user" set.
    Context "kubectl" modified.
    Switched to context "kubectl".

    [cloudusr@j scripts]$ kubectl get pods --all-namespaces
    NAMESPACE     NAME                                                      READY     STATUS              RESTARTS   AGE
    kube-system   auth-apikeys-29j0b                                        1/1       Running             0          11m
    kube-system   auth-idp-m7grf                                            3/3       Running             0          13m
    kube-system   auth-pap-2rx27                                            1/1       Running             0          11m
    kube-system   auth-pdp-9scjn                                            1/1       Running             0          11m
    kube-system   calico-node-amd64-k31w5                                   2/2       Running             0          21m
    kube-system   calico-node-amd64-lbt1f                                   2/2       Running             0          21m
    kube-system   calico-node-amd64-r1l1f                                   2/2       Running             0          21m
    kube-system   calico-policy-controller-1048521425-21j6h                 1/1       Running             0          21m
    kube-system   catalog-catalog-apiserver-gdhls                           0/1       ContainerCreating   0          11m
    kube-system   catalog-catalog-controller-manager-3100032879-gl3mg       0/1       ContainerCreating   0          10m
    kube-system   catalog-ui-wbzvm                                          1/1       Running             0          11m
    kube-system   default-http-backend-198681862-rx2sn                      1/1       Running             0          11m
    kube-system   elasticsearch-client-3479638665-q9dcx                     0/2       PodInitializing     0          11m
    kube-system   elasticsearch-data-0                                      0/1       PodInitializing     0          11m
    kube-system   elasticsearch-master-1570256108-gmfrn                     0/1       PodInitializing     0          11m
    kube-system   filebeat-ds-amd64-3ph6r                                   1/1       Running             0          11m
    kube-system   filebeat-ds-amd64-4kkgm                                   1/1       Running             0          11m
    kube-system   filebeat-ds-amd64-s0h34                                   1/1       Running             0          11m
    kube-system   heapster-1250025240-lf9dj                                 1/2       CrashLoopBackOff    5          11m
    kube-system   helm-api-792621620-r78vb                                  0/1       ContainerCreating   0          9m
    kube-system   helmrepo-2349593664-b3lc6                                 0/1       ContainerCreating   0          9m
    kube-system   icp-ds-0                                                  1/1       Running             0          18m
    kube-system   icp-router-ct1lz                                          0/1       ContainerCreating   0          9m
    kube-system   image-manager-0                                           0/2       ContainerCreating   0          11m
    kube-system   k8s-etcd-10.10.25.157                                     1/1       Running             0          20m
    kube-system   k8s-mariadb-10.10.25.157                                  1/1       Running             0          20m
    kube-system   k8s-master-10.10.25.157                                   3/3       Running             0          20m
    kube-system   k8s-proxy-10.10.25.157                                    1/1       Running             0          19m
    kube-system   k8s-proxy-10.10.25.158                                    1/1       Running             0          20m
    kube-system   k8s-proxy-10.10.25.159                                    1/1       Running             0          19m
    kube-system   kube-dns-1038623989-jv1hd                                 3/3       Running             0          19m
    kube-system   logstash-4245234969-pv82n                                 1/1       Running             0          11m
    kube-system   metering-dm-653503904-q63wq                               1/1       Running             0          11m
    kube-system   metering-reader-amd64-3lg3d                               1/1       Running             0          11m
    kube-system   metering-reader-amd64-kwphq                               1/1       Running             0          11m
    kube-system   metering-reader-amd64-l2lkz                               1/1       Running             0          11m
    kube-system   metering-server-1157299371-6cmhj                          1/1       Running             0          11m
    kube-system   metering-ui-3174349629-36c0b                              0/1       ContainerCreating   0          11m
    kube-system   monitoring-exporter-3293343457-fvdbs                      0/1       ContainerCreating   0          10m
    kube-system   monitoring-grafana-2476181526-fzp7z                       0/2       ContainerCreating   0          10m
    kube-system   monitoring-prometheus-2422752664-pmmbn                    0/2       ContainerCreating   0          10m
    kube-system   monitoring-prometheus-alertmanager-3766402458-wfq4d       0/2       ContainerCreating   0          10m
    kube-system   monitoring-prometheus-kubestatemetrics-3025976457-bjnxk   0/1       ContainerCreating   0          10m
    kube-system   monitoring-prometheus-nodeexporter-amd64-5f5kw            0/1       ContainerCreating   0          10m
    kube-system   monitoring-prometheus-nodeexporter-amd64-b101z            1/1       Running             0          10m
    kube-system   monitoring-prometheus-nodeexporter-amd64-ll4wm            1/1       Running             0          10m
    kube-system   nginx-ingress-lb-amd64-8vs8w                              1/1       Running             0          11m
    kube-system   platform-api-pzb2k                                        1/1       Running             0          11m
    kube-system   platform-ui-sdmb4                                         1/1       Running             0          11m
    kube-system   rescheduler-w83vj                                         1/1       Running             0          11m
    kube-system   tiller-deploy-2307655136-881r5                            1/1       Running             0          11m
    kube-system   unified-router-86zn3                                      1/1       Running             0          11m
  9. View the Web UI

    Upon successful completion of the¬†08-install.sh script,¬†instructions on how to access the web interface will be printed. if the UI does not appear, you may need to wait for your pods to be¬†in the “Running” state. By default the web interface will be accessible on https://<PUBLIC_IP>:8443¬†with the username/password of admin/admin. The cluster is using a self-signed certificate, so you may get a message about an untrusted certificate. Accept and proceed to the log in page.¬†Log in and view the dashboard. Many elements of Kubernetes that can be accesses via kubectl commands can also be viewed using the left-hand navigation menu. Additional options such as LDAP configuration and a catalog of content can also be found here.

    Self-signed certificate warning, click¬†“advanced” and proceed to the page.

    self-signed-cert

    Accepting the warning will lead you to the log in page, log in:

    log-in-page

    View the dashboard and explore the UI

    dashboard

  10. Conclusion

    IBM Cloud Private is a fully-featured Kubernetes-based platform that runs on your infrastructure. The Community Edition is a great way to test out the platform. The scripts that accompany this article are intended to provide a digestable way to understand the install process and get the product up-and-running on your machines, so you can do the important tasks using the platform.

11 comments on"IBM Cloud Private on RHEL"

  1. can this be adapted to install on PPC64LE ?

    • Jesse Antoszyk November 27, 2017

      Hey @CMcGee,

      Thanks for the feedback. I have created a branch for ppc64le support that I believe should work. Once I can get a ppc machine, I will verify the changes and merge to master. If you would like to help me test this branch, you can change the clone (step 3) to:
      git clone -b ppc-support https://github.com/jcantosz/icp-rhel-install.git
      all the other steps should be the same. I appreciate the help!

      Thanks,
      ~Jesse

      • Awesome Jesse, I will be looking into this over the next week.

        • Jesse Antoszyk November 29, 2017

          Great to hear! I look forward to your feedback. I think I am close to getting a ppc machine for testing, will update the scripts if I find bugs ūüôā

          • Jesse Antoszyk December 05, 2017

            I have merged the PPC branch into master, no need to clone -b anymore. The original instructions should work for ppc64le and x86_64

    • DigitalHealthProgram October 22, 2018

      Hello,

      Can this be installed on a oracle Virtual Box for MAC

  2. Curious enough but you use Docker-CE on RHEL in 03-install-packages.sh.
    Can I think that IBM supports the environment running Docker-CE(for CentOS) on RHEL because of this post, with using ICP’s commercial edition?

    • Jesse Antoszyk November 29, 2017

      Hey @sousatou,

      I would consider this recipe and the GitHub repo and unofficial/unsupported route for installing IBM Cloud Private.
      I do not know IBM’s official support policy for docker ce on RHEL. I will see if I can find a concrete answer for you on that topic.

      Thanks,
      ~ Jesse

  3. Thanks for your reply.
    Preparing docker engine for RHEL/bootnode is the biggest pain to use ICP. So I’m just curious how anyone solve the matter.
    But the article helps a lot to try using ICP on RHEL. Thanks for the post!

    • Jesse Antoszyk November 30, 2017

      Hey @sousatou,
      Got some more info from the ICP team. Starting in the next release (2.1.0.1), ICP will provide a Docker package for RHEL. Support of this Docker package will be limited to ICP usage.
      Thanks!

      • Jesse,
        It’s truly a great news! Now I don’t need to recommend to use Ubuntu only for boot node even though it’s production environent.
        Thanks, really.

Join The Discussion