POWER-Up toolkit automates the deployment of baremetal OpenPOWER and x86 server clusters. It¬† automates the network setup by taking pre-existing configurations from a network switch and configures the nodes utilizing Ansible and Cobbler. POWER-Up supports Ubuntu, CentOS, and RHEL. POWER-Up also configures the client and network switch saving a user time configuring a new node. Utilizing POWER-Up’s robust features we can install IBM Cloud Private (ICP) in order to fully automate a cluster installation in a data center or small server room.

What is Cluster POWER-Up?

Cluster POWER-Up is an Ansible and Python based bare-metal hardware deployment tool.¬† By using POWER-Up‚Äôs interface and node templates, you can easily deploy complex heterogeneous clusters on your bare-metal OpenPower systems. POWER-Up configures multiple types of switches from different switch vendors. This configuration is done by using an editable YAML text file that drives the cluster configuration. The configuration process is driven from a ‚Äúdeployer‚ÄĚ node that can be removed from the cluster when the setup is finished.¬†

Cluster POWER-Up hardware prerequisites

  1. Properly rack and connect necessary cables to the hardware.
  2. Initialize hardware: Initialize switches with static IP address, userid and password.

    Ensure that all cluster compute nodes are set to obtain a DHCP address on their BMC ports and that they are configured to support PXE boot on one of their network adapters.

The POWER-Up process is as follows: 

  1. Install the Cluster POWER-Up software on the deployer node. Read the docs here.
  2. Edit an existing configuration.yaml file to drive the configuration. A sample configuration.yaml file can be found in the POWER-Up github repo. Users should edit this YAML file to include their personal network settings, user information, etc.
  3. Run the POWER-Up software

When finished, Cluster POWER-Up generates a YAML-formatted inventory file with detailed information about your cluster nodes and is made available on the base power-up directory. This file can be read by operational management software and used to seed configuration files that are needed when installing a solution software stack. An example inventory.yaml file can also be found on the ICP and POWER-Up github repo..

Installing IBM Cloud Private Community Edition

Before setting up your files, see the ICP resource requirements for 3.1.2 CE. POWER-Up’s commandline tool is called pup which will be mentioned throughout this post. This post utilizes scripts and examples hosted on the ICP POWER-Up github repo.

Customizing the Configuration for a Cluster

The configuration.yaml file contains a lot of configuration information for pup. To enable a cluster for a specific environment, edit the provided ICP sample configuration.yaml file with the proper configuration parameters that pertain to your environment. For example,

~ ~ ~ ~ ~ ~ ~  licensing comment and YAML ~ ~ ~ ~ ~
switches:
    mgmt:
        - label: mgmt1
          class: lenovo
          userid: admin          <-- Type your management switch user id here.         
          password: admin          <--  Type your management switch password here.
          interfaces:
              - type: inband
                ipaddr: 192.168.3.5          <--  Type your management switch IP address here.
                port: 1
data:
        - label: data1
          class: Lenovo
          userid: admin          <--  Type your data switch user id here.         
          password: admin          <--  Type your data switch password here.
          interfaces:
              - type: inband
                ipaddr: 1.2.3.178          <--  Type your data switch IP address here.
~ ~ ~ ~ ~ ~ ~  YAML and comments ~ ~ ~ ~ ~
interfaces:
    - label: ipmi
      description: pxe interface
      iface: eth0
      method: dhcp
 
    - label: pxe
      description: pxe interface
      iface: eth0
      method: dhcp
 
    - label: external
      description: Organization site or external network
      iface: eth1
      method: static
      address_list:
          - 1.2.3.18-1.2.3.20          <-- Type your client IP here().
          - 1.2.3.38          <--  Type your client IP here.
          - 1.2.3.76          <-- Type your client IP here.
      netmask: 255.255.255.0          <--  Type your netmask IP here.
      broadcast: 1.2.3.255          <--  Type your broadcast IP here.
      gateway: 1.2.3.1          <--  Type your gateway IP here.
      dns_search: aus.stglabs.ibm.com
      dns_nameservers: 1.2.3.200          <--  Type your nameserver IP here.
 
    - label: interconnect
      description: Private 10G Data Network to Interconnect Cluster
      iface: eth2
      method: static
      address_start: 10.0.0.1
      netmask: 255.255.255.0
      broadcast: 10.0.0.255
 
networks:
    external:
        description: Organization site or external network
        addr: 1.2.3.4/24                      <--  Type your subnet address here.
        broadcast: 1.2.3.255                   <-- Type your broadcast IP here.
        gateway: 1.2.3.1                      <--  Type your gateway IP here.
        dns-nameservers: 1.2.3.4       <-- Type your nameserver IP here.
        dns-search: aus.stglabs.ibm.com
        method: static
        eth-port: eth10
    interconnect:
        description: Private 10G Data Network to Interconnect Cluster
        addr: 10.0.0.0/24
        broadcast: 10.0.0.255
        method: static
        eth-port: eth11
 ~ ~ ~ ~ ~ ~ ~  bunch of YAML and comments ~ ~ ~ ~ ~
node-templates:
    controller1:
        hostname: min                        <--  Type your hostname here.
        userid-ipmi: ADMIN                <--  Type your userid here.
        password-ipmi: admin            <--  Type your password here.
        cobbler-profile: ubuntu-16.04.1-server-ppc64el
  ~ ~ ~ ~ ~ ~ ~ bunch of YAML and comments ~ ~ ~ ~ ~ 

Using POWER-Up to Install ICP

  1. Create a configuration.yaml This file is responsible for driving the installation and will eventually call the scripts to install ICP during the post-installation. You can look at an example configurations on the POWER-Up github repo.
  2. Uniquely identify a single node We must install the ICP Master software on a single node in our cluster. To do so, we provide a unique hostname to each node. In this example, the ICP master node will be server-1. The hostname is defined in the inventory.yaml. See an example of an inventory.yaml on the ICP and POWER-Up github repo.
  3. Create a post-installation action (software-bootstrap) The final step is defining the software-bootstrap section at the bottom of the configuration.yaml. This step is executed once the installation of our Operating System is complete. This is the step where pup will install ICP.

Look at this small configuration.yaml example. I will further discuss these bash commands at the bottom of this post. To summarize, the bash commands from this snippet will execute bash scripts which are responsible for installing ICP. 

Installing ICP CE on RHEL with POWER-Up

  1. Add your details for Red Hat’s subscription manager into your configuration.yaml. An example is provided below:

    node_templates:
        - ...
          os:
              hostname_prefix:
              profile:
              install_device:
              redhat_subscription:
                  state: present
                  username: joe_user
                  password: somepass
                  pool_ids:
                    - 0123456789abcdef0123456789abcdef
                    - 1123456789abcdef0123456789abcdef
              users:
                  - name:
                    password:
              groups:
                  - name:
              kernel_options:

  2. Add the software-bootstrap section to the bottom of your configuration.yaml. This section is responsible for installing ICP on the cluster.

    software_bootstrap:  this is the title of the section in the YAML file which will execute once we call pup post-deploy
       - hosts: server-1  execute this script (install ICP) only on the server-1 node, which is the hostname of the node
         command: |  defining a bash command to execute on server-1
         wget https://raw.githubusercontent.com/open-power-ref-design/ICP-install/master/scripts/icp-install-rhel.sh 
          chmod +x icp-install-rhel.sh
         ./icp-install-rhel.sh {{ icp-version }} 

    The above section in red is simply downloading a bash script onto the client. This bash script is responsible for installing ICP CE. This action will only be completed on server-1, our ICP Master node. 

  3. Provision our machines and install ICP Master on a single node. Run the following command:

    pup deploy --extra-vars 'icp-version=3.1.2' configuration.yaml

    After the pup deploy command is finished, your ICP Master will be available at https://your_ip:8443.

Installing ICP CE on Ubuntu with POWER-Up

  1. Add the software-bootstrap section to the bottom of your configuration.yaml. This section is responsible for installing ICP on the cluster.

    software_bootstrap:  this is the title of the section in the YAML file which will execute once we call pup post-deploy
       - hosts: server-1  execute this script (install ICP) only on the server-1 node, which is the hostname of the node
          command: |  defining a bash command to execute on server-1
          wget https://raw.githubusercontent.com/open-power-ref-design/ICP-install/master/scripts/icp-install-debian.sh 
          wget https://raw.githubusercontent.com/open-power-ref-design/ICP-install/master/scripts/install_docker.sh 
          chmod +x install_docker.sh
          chmod +x icp-install-debian.sh
          ./icp-install-debian.sh {{ icp-version }} 

    The above section in red is simply downloading two bash scripts onto the client. One bash script is responsible for installing Docker and the other script will install ICP CE. This action will only be completed on server-1, our ICP Master node. 

  2. Provision our machines and install ICP Master on a single node. Run the following command:

    pup deploy --extra-vars 'icp-version=3.1.2' configuration.yaml

After the pup deploy command is finished, your ICP Master will be available at https://your_ip:8443.

This post describes installing ICP on a single node, server-1. The following section is an example of a bash command which can be executed on all hosts, not just server-1.
   

 - hosts: all
      command: touch xyz.txt for all the nodes on the network, add a file called xyz.txt (just an example). 

If you have any issues with pup, you can file a bug on the POWER-Up repo.

Thanks for reading!

Join The Discussion

Your email address will not be published. Required fields are marked *