IBM Z Day on Nov. 21: Discover the ideal environment for modern, mission-critical workloads. Learn more

Configure a CI/CD pipeline with Jenkins on Kubernetes

DevOps encourages collaboration, cooperation, and communication between developers and operations teams to improve the speed and quality of software development. One of the key principles of DevOps is automation, which reduces human error, provides consistent results, and even mitigates risks. With the help of automation, you and your team can build, test, and deploy software quickly and efficiently.

Learning objectives

In this tutorial, you’ll learn how to achieve a fully automated environment with Jenkins on Kubernetes by learning how to:

  • Set up a Jenkins environment on Kubernetes
  • Configure a CI/CD Jenkins pipeline
  • Build Docker images using Jenkins
  • Push Docker images to a Docker registry
  • Deploy Docker images to a Kubernetes environment
  • Integrate Slack and Jenkins
  • Integrate GitHub and Jenkins using GitHub Webhooks

Prerequisites

$ curl -sL https://ibm.biz/idt-installer | bash

Before we begin creating, building, and deploying, let’s first understand some of the tools and software we’ll be using throughout this tutorial.

What is Jenkins?

Jenkins is an open source automation server that you can use to automate tasks related to software development, testing, or deployments.

What is Docker?

Docker is an open source software platform built to make it easier to create, deploy, and run applications using containers. A container is a standardized package of software, which can include libraries and other dependencies.

What is Kubernetes?

Kubernetes is an open source container orchestration platform for automating deployment and scaling and managing containerized workloads and services.

Now we’re ready to get started. Feel free to follow along with my sample code here.

Step 1. Create a Kubernetes cluster on IBM Cloud

Login to IBM Cloud and choose the Kubernetes option from the navigation menu. Then, click on the Create Cluster button and select the Free plan. This process will take approximately 40 minutes.

image

Step 2. Build a modified Jenkins image

Early on, Jenkins was designed to run on physical machines without any containerization technology. As containerization become more popular, Jenkins adapted its solution to the new containerized world. Unfortunately, this change brought some challenges. For instance, Jenkins requires Docker to build Docker images. However, a containerized version of Jenkins does not contain Docker and Docker CLI by default. For this reason, a new Docker image that contains Docker CLI and other tools must be created by using a Jenkins image as a base image. To do this, Docker uses Dockerfile to build custom images. In the modified-jenkins directory there is a Dockerfile to build a custom modified Jenkins image. I’ll break this down, step-by-step.

The base image should look something like this:

FROM jenkins/jenkins:lts

The command line that comes after, USER root, grants root access to the image, which will be used while installing the Docker CLI and Kubernetes CLI (kubectl).

ENV DOCKERVERSION=18.03.1-ce sets an environment variable.

The following command line downloads Docker, installs the Docker CLI, and removes the Docker Daemon:

RUN curl -fsSLO https://download.docker.com/linux/static/stable/x86_64/docker-${DOCKERVERSION}.tgz \
  && tar xzvf docker-${DOCKERVERSION}.tgz --strip 1 \
                 -C /usr/local/bin docker/docker \
  && rm docker-${DOCKERVERSION}.tgz

Note that the mofidied Jenkins image does not contain Docker Daemon. Docker Daemon will run in another container and the modified Jenkins image will refer to that Docker Daemon container. Check out Docker in Docker for more information.

The following downloads the Kubernetes CLI (kubectl) and installs it:

RUN curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.t)/bin/linux/amd64/kubectl \
    && chmod +x ./kubectl \
    && mv ./kubectl /usr/local/bin/kubectl

Once finished, your code should look something like this:

FROM jenkins/jenkins:lts

USER root

ENV DOCKERVERSION=18.03.1-ce

RUN curl -fsSLO https://download.docker.com/linux/static/stable/x86_64/docker-${DOCKERVERSION}.tgz \
  && tar xzvf docker-${DOCKERVERSION}.tgz --strip 1 \
                 -C /usr/local/bin docker/docker \
  && rm docker-${DOCKERVERSION}.tgz

RUN curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl \
    && chmod +x ./kubectl \
    && mv ./kubectl /usr/local/bin/kubectl

Next, open a terminal and navigate to the modified-jenkins directory:

cd modified-jenkins

In the next command line, replace kmlaydin with your Docker Hub username to build a modified Jenkins image:

docker build -t kmlaydin/modified-jenkins:latest

image

Congratulations! The modified Jenkins image was successfully built.

Step 3. Deploy a modified Jenkins image to Kubernetes

The modified Jenkins image is in the local environment now and Kubernetes cannot access the local images. This is where Docker Hub comes into the picture. Docker Hub is a cloud-based repository in which Docker users and partners create, test, store, and distribute container images. A modified Jenkins image is needed to push to the Docker Hub or other container registries like IBM Cloud Container Registry. By default, Docker uses Docker Hub.

  1. First, start by running the following command to login to Docker Hub via the terminal:

    docker login

  2. Next, push the modified Jenkins image to Docker Hub (don’t forget to replace kmlaydin with your Docker Hub username):

    docker push kmlaydin/modified-jenkins:latest

    image

    The pushed image can now be seen via Docker Hub and Kubernetes can now access the image conveniently. Kubernetes works with YAML files to handle these configurations.

  3. Now you need to open the jenkins-deployment.yaml file, located in the modified-jenkins directory, with a code editor.

  4. Find kmlaydin/modified-jenkins:latest and replace the Docker Hub username, pushed image’s name, and version.

    image

    The deployment file is now ready to deploy modified Jenkins to Kubernetes.

  5. Now open the Kubernetes cluster dashboard which we created in step 1. Navigate to the access tab and run the commands to gain access to the Kubernetes cluster via the terminal.

    image

  6. Make sure that the directory is modified-jenkins and run the following commands to deploy the modified Jenkins to Kubernetes:

     kubectl apply -f jenkins-deployment.yaml
     kubectl apply -f jenkins-service.yaml
    
  7. Now run the kubectl get deployment,pod,service command to make sure that Jenkins is deployed and has a running status. The deployment process may take a couple of minutes.

    image

  8. Retrieve the external IP of your worker node to gain access to the Jenkins dashboard:

     $ export EXTERNAL_IP=$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type=="ExternalIP")].address }')
    
     $ export NODE_PORT=30100
    
     $ echo $EXTERNAL_IP:$NODE_PORT
    

    image

Step 4. Set up a Jenkins environment

Now it’s time to set up your Jenkins environment. Jenkins assigns the initial admin password, which can be found by logging the Jenkins container.

  1. First, obtain the logs of the Jenkins container:

    kubectl logs $(kubectl get pods --selector=app=jenkins -o=jsonpath='{.items[0].metadata.name}') jenkins

    The initial admin password can be found between the rows of asterisks.

  2. Now select the Install suggested plugins option. Continue by selecting Continue as admin and then click the Save and Finish button.

    And just like that, Jenkins is ready to use.

    image

    With Jenkins ready for use, the following credentials are needed for Jenkins to fulfill a duty conveniently and to run properly:

    • GitHub credentials to gain access to source code
    • Docker Hub credentials to push a built image to Docker Hub
    • A Kubeconfig file to gain access to a Kubernetes cluster

      GitHub and Docker Hub credentials are a type of Username with a password. A Kubeconfig credential is a type of Secret file. Remember, in step 3, you downloaded the Kubeconfig file and it was exported as an environment variable to your local computer.

  3. Now you need to go to the Kubeconfig directory of the Kubernetes cluster (the Kubeconfig environment variable should already be set before running this command):

    $ cd $(echo $KUBECONFIG | awk '{split($0,a,"kube-config"); print a[1];}') && ls

    There should be two files in the directory:

    • The <PEM-FILE>.pem file stands for Privacy-Enhanced Mail and is a file format for storing and sending cryptographic keys, certificates, and other data.
    • The <KUBE-CONFIG>.yml file is used to configure access to a cluster and sometimes is called a kubeconfig file, which is a generic way of referring to configuration files.
  4. Both files should be in the same directory, but in Jenkins, there is no option for that. For this reason, the <PEM-FILE>.pem file should be embedded into the <KUBE-CONFIG>.yml file. To do this, copy both files in to the desktop directory. The copying process is not obligatory but is done to preserve the original files. Now run this command in the directory that contains the <PEM-FILE>.pem file and the <KUBE-CONFIG>.yml file:

    $ for file in ./*; do cp $file /Users/$USER/Desktop; done;

    If you’d like, the destination directory can be changed by editing /Users/$USER/Desktop.

  5. Next, go to the desktop directory via the terminal. Encode the <PEM-FILE>.pem file as base64:

    $ base64 <PEM-FILE>.pem

  6. Copy the result and open the <KUBE-CONFIG>.yml file with a code editor. Find certificate-authority: <PEM-FILE>.pem and change it to certificate-authority-data: <BASE64-RESULT>.

    image

    After completing the steps above, the <KUBE-CONFIG>.yml now contains the <PEM-FILE>.pem file.

  7. Now go back to the Jenkins dashboard and find the Credentials option in the left pane and select the (global) option. The credentials can be added by clicking the Add Credentials button in the left pane. First, add the GitHub credentials as Username with password with the ID github. Then add the Docker Hub credentials as Username with password with the ID dockerhub. Lastly, add the Kubeconfig credentials as Secret file with the ID kubeconfig.

    The credentials are now ready to use and plugins need to be installed. Be aware that Jenkins has a wide range of plugin options. The Kubernetes CLI plugin is not mandatory; however, it eases the process. Kubernetes CLI allows you to configure kubectl to interact with Kubernetes clusters.

  8. Let’s get that set up by going back to the Jenkins dashboard and finding the Manage Jenkins option in the left pane. Select Manage Plugins and then choose the Available tab. There should be a lot of available plugins in that tab. Search for the Kubernetes CLI plugin and install it. Hooray, Jenkins is ready for the first landing!

Step 5. Create the first Jenkins pipeline

To create the first Jenkins pipeline, a GitHub account that contains example files is needed. Example files can be deployment.yaml, Dockerfile, index.js, Jenkinsfile, or service.yaml.

Navigate back to the Jenkins dashboard and find the New Item option in the left pane. Enter an item name and choose Pipeline. An example project URL is https://github.com/kemallaydin/jenkins-example.

Choose GitHub Project and type your project’s URL. Find the pipeline section and change the definition value from Pipeline script to Pipeline script from SCM. For the SCM option, choose Git.

Now type in your repository URL and choose the GitHub credentials.

image

Great! You’re now ready for the next step.

Step 6. Integrate Jenkins and Slack

To post notifications to a Slack channel, Jenkins needs the Slack Plugin. Go back to the Jenkins dashboard and find the Manage Jenkins option in the left pane. Select the Manage Plugins option and choose the Available tab. Search for the slack notification plugin and install it.

Hopefully you’ve already created your Slack account. If not, click here to get started. Once you have a Slack account, configure the Jenkins integration by using Jenkins CI. After configuration, click on Manage Jenkins again in the left navigation and go to Configure System. Find the Slack section and add the following values:

  • Workspace: <TEAM-SUBDOMAIN>
  • Credential: <INTEGRATION-TOKEN-CREDENTIAL-ID>
  • Default channel / member id: <CHANNEL-NAME>

If you’d like, you can create a secret text credential by clicking the Add button. You can also test the Jenkins and Slack integration by clicking the Test Connection button.

image

Step 7. Integrate Jenkins and GitHub

To receive specified events to Jenkins from GitHub, you need to configure Webhooks. Webhooks allow external services to be notified when certain events happen. When the specified events happen, GitHub will send a POST request to Jenkins.

To begin configuration, navigate to your project repository on GitHub. Click Settings in the right corner and find the Webhooks option in the left pane. Click the Add Webhook button. The payload URL is http://<JENKINS-URL>:<JENKINS-PORT>/github-webhook/ and an example URL is http://169.47.252.31:30100/github-webhook/. Once finished, make sure you save the Webhook.

Jenkins is now configured to accept events from GitHub and there’s only a few steps left to complete the Jenkins and GitHub integration.

Head over to the first pipeline’s dashboard and click the Configure option. Choose GitHub hook trigger for GITScm polling under the Build Triggers section and save the configuration. The pipeline should be triggered manually once to identify stages which are used in the Jenkinsfile. After that, the GitHub Webhook can trigger the pipeline.

image

Step 8. Test the first Jenkins pipeline

You’ve made it to the final step! Jenkins is finally ready to test. Go to the first pipeline’s dashboard and click Build Now. The steps that are defined in Jenkinsfile are now available. Simply make a small change on the index.js and push it to GitHub. The pipeline is triggered by GitHub and Jenkins deploys a sample NodeJS application to Kubernetes. The application can be found at http://<JENKINS-URL>:30300/getpodinfo.

image

Lastly, run kubectl get pods -w to watch the changes in Kubernetes while the building process is running.

image Note: In this demonstration, the replica size is increased from 1 to 10. To test it, modify the deployment.yaml.

Summary

Congratulations! You’ve successfully set up a continuous integration environment using Jenkins, Slack, and GitHub. Integrating GitHub with Jenkins automates deployment, testing, and ensures your projects are always up to date.

To explore more of what you can do, check out IBM Cloud Kubernetes Service and Jenkins.

Kemal Aydin