Red Hat Openshift Container Platform version 4, when deployed using a User Provisioned Infrastructure (UPI), offers many challenges for the average system administrator. I created a guide that covers the basics of deploying OpenShift on IBM Z/LinuxONE hardware using a UPI.

However, one of the items that I did not cover in detail was the internal network part of OpenShift from the install-config.yaml configuration file, which is a key part of the UPI installation method. So that's what I will cover in this brief tutorial.

Learning objectives

This tutorial can help you improve your understand of how the OpenShift Software Defined Network (SDN) works. It shows you how to configure the OpenShift SDN, and uses examples to explore the network section of the install-config.yaml.

Prerequisites

To complete this tutorial, you need an intermediate knowledge of networking as we will be covering software-defined networks as well as CIDR blocks, subnets, and network package encapsulation (VxLAN). I also recommend having an intermediate knowledge of the OpenShift architecture.

The OpenShift Container Platform uses a software-defined networking (SDN) approach to provide a unified cluster network that enables communication between pods across the OpenShift Container Platform cluster. This pod network is established and maintained by the OpenShift SDN, which configures an overlay network using Open vSwitch (OVS).

Figure 1. Pod-to-pod traffic on two different nodes

Pod-to-pod traffic

Estimated time

It should take you about 15 minutes to complete this tutorial.

Steps

During the OpenShift installation process using the UPI approach, one of the checklist items is to edit the install-config.yaml file. This file contains many sections, but we will only be working with the networking section.

OpenShift uses two types of networks -- an external network and an internal network (SDN). The following sections cover how the internal network is configured and all the details on how to tweak this configuration if needed.

Figure 2. OpenShift networks

OpenShift networks

Let's start with a sample install-config.yaml file:

apiVersion: v1
baseDomain: <domain>
compute:
- architecture: s390x
  hyperthreading: Enabled
  name: worker
  replicas: 0
controlPlane:
  architecture: s390x
  hyperthreading: Enabled
  name: master
  replicas: 3
metadata:
  name: <cluster_name>
networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  networkType: OpenShiftSDN
  serviceNetwork:
  - 172.30.0.0/16
platform:
  none: {}
fips: false
pullSecret: '<pull-secret>'
sshKey: '<ssh-public-key>'

Now let's focus on just the networking section of this configuration file, which is divided in two major parts. The first is clusterNetwork:

clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23

clusterNetwork defines the block of IP addresses that will be used for all of the created Pods within your OpenShift cluster. (Note: This block must not overlap with existing physical networks.)

But, what is hostPrefix: 23? That is the subnet prefix length that's assigned to each individual node. For example, if hostPrefix is set to 23, then each node is assigned a /23 subnet out of the given cidr, which allows for 510 (2^(32 - 23) - 2) pod IPs addresses.

Let's fact check the statement above by using the web console and by querying some of the workloads I have on my OpenShift cluster:

From the left menu, select Pods and then select the liberty-1-qwqpt pod. Verify that the IP of this pod is 10.128.2.12:

Figure 3. Selecting the pod

Selecting the pod

Note that the Pods are using the same IP range that's defined by the CIDR configuration from the clusterNetwork section of the install-config-yaml file.

Now take a look at the next part of the networking section:

networkType: OpenShiftSDN
  serviceNetwork:
 - 172.30.0.0/16

Fact check time! Using the OpenShift web console, from the lefthand menu select Networking -> Services and select the Liberty service. Verify that the IP for this service is 172.30.115.26.

Figure 4. Verifying the service IP

Verifying the service IP

What if you want to access this application from outside your cluster? Simple: From the left menu, select Routes -> Create Route. Then set a name for the route, select a service and a target port, and then click the Create button as shown in Figure 5:

Figure 5. Creating a route

Creating a route

Once a route is created, the fully qualified domain name (FQDN) of your application appears under Location as a hyperlink that, when clicked, will open your default browser and allow you to access your application from outside your cluster.

Figure 6. Fully qualified domain name

Fully qualified domain name

You might be wondering what IP of this route is. In fact, it's the same IP as that of your load balancer as defined by your DNS infrastructure. (Remember that?)

Figure 7 is a visual representation of the OpenShift network (not the internal cluster network):

Figure 7. OpenShift network

OpenShift network

Now it's time to drill down and look at a visual representation of the OpenShift internal cluster -- but before we do that, let's scale our application to 2 pods to make things more interesting. Select Deployment Configs in the left menu, select the deployment config name for your application (in this case, liberty), then under the Overview tab scale to 2 pods using the arrows below Deployment Config Overview:

Figure 8. Scaling an application

Scaling an application

You can verify that the pod was scaled to 2 pods by selecting Pods from the left menu:

Figure 9. Verifying that pods have been scaled

Verifying that pods have been scaled

Now, select the second pod and verify its IP address:

Figure 10. Verifying IP address of second pod

Verifying IP address of second pod

You'll see that the new pod has been assigned the IP address 10.131.0.46, which is within the range you specified in your configuration file.

Figure 11 gives a visual representation of the service IP and the IP addresses for the two pods:

Figure 11. Service IP and pods' IP addresses

Service IP and pods' IP addresses

Summary

This tutorial has showed you how to define the cluster internal IP ranges defined in the networking section of the install-config.yaml file, and demonstrated through examples how pods and services benefit from this.

Note that the IP addresses that you define for the actual nodes in your OpenShift cluster during the Red Hat CoreOS installation process -- for example master0, master1, worker0, and worker1 -- will have different IP ranges than the those used internally by OpenShift (for both clusterNetwork and serviceNetwork) as they will use an IP range for external networks

Acknowledgements: Thanks to Brad Hinson and Shanna Chan, who contributed to this tutorial.

Start working with Red Hat OpenShift 4 networks

Start working with Red Hat OpenShift 4 networks