Overview

Skill Level: Any Skill Level

Familiar with GitHub, Travis CI, Ansible, and Bluemix

Application development with agile coding practices and traditional release cycles are a reality for many teams. This recipe explains how to accommodate a split CI and CD pipeline using GitHub, Travis CI, and Ansible.

Ingredients

This is the second recipe in a series. For the first segment, check the out how to Use Travis CI to Continuously Deploy your Applications to IBM Bluemix.


In this tutorial we will create an application repository to house our code, and a deployment repository to store our deployment logic. The application repository will generate build artifacts which the deployment repository will consume and deploy. The artifacts are deployed using an Ansible Playbook. Ansible is a simple, powerful, agentless technology for automating application deployment and configuration management. Ansible playbooks can be used to deliver simple applications as is this recipe as well as complex application infrastructure.

Step-by-step

  1. Generate Token in GitHub

    1. In GitHub, click on the dropdown next to your gravitar and select Settings.
      GitHub Settings Navigation
    2. Under the Developer settings section select Personal access tokens.
      GitHub Personal Access Token
    3. Click the Generate new token button.
      GitHub Generate new Token
    4. Give your token a description, select the scopes, then click Generate token. For this recipe you will need the public_repo scope.
      GitHub Token Settings
    5. Copy this token for later use.
  2. Fork the application repo in GitHub

    1. Fork IBMCloudDevOps/bluemix-python-app-sample. This will be our application repository. 
      Github Fork Repo

    The separation of the application source (what is does) from the deployment logic (where it runs) is reflected in our repositories. This repository contains the application source. You will fork the deployment repository in step 6.

  3. Setup Travis CI to build your application repo

    1. Sign into Travis CI with your GitHub ID.
      Travis Sign In
    2. Click the + next to My Repositories to enable Travis CI to build the new repository.
      Travis Add Repo
    3. Flick the repository switch on for your repository. Sync account if the repository does not appear in your list.
    4. Click the gear icon beside your repository to ‚ÄúGo to Travis CI repository settings‚ÄĚ.
      Travis Enable Repo and Settings Cog
    5. Set the following Travis CI environment variables (ensure Display value in build log is off to protect your information):
      • GITHUB_OAUTH_TOKEN – The token we generated in step 1. This will allow us to create GitHub releases as output from our CI pipeline¬†
        Travis Environmental Variables
  4. Update manifest.yml to provide a unique hostname

    If you are using the bluemix-python-app-sample, switch to the branch updateManifest. Then replace the host field <APP_HOSTNAME> in the manifest.yml with a unique host name in the mybluemix.net domain.

    applications:
    - path: .
    memory: 256M
    instances: 1
    domain: mybluemix.net
    name: test-python
    host: <APP_HOSTNAME>
    disk_quota: 1024M

     

    The manifest.yml file is used to configure your application to Bluemix. Using a duplicate hostname on the mybluemix.net domain will cause the deploy to fail with the message:

    Server error, status code: 400, error code: 210003, message: The host is taken:

    Commit and push the manifest.yml updates to your repository.

  5. Create a pull request on your application repository

    1. Create a new pull request from the branch updateManifest to the master branch of your base repository. Make sure you are not pulling against the upstream master. GitHub Create Application Pull Request
    2. Wait for the checks to pass and merge the pull request
  6. Application repository travis build

    1. Go to your repo in Travis CI and watch the build for the master branch run. This will deploy your artifacts as a GitHub Release. Tagging a release in GitHub will cause another Travis CI build to run. This build will only re-run the linter (not create another release) and thus can be ignored.
      Travis Master Branch Build
    2. Once Travis CI has completed, take a look at the GitHub releases for your repo. You should see a new release with a bluemix-python-starter.tz download link.
      GitHub Releases Tab
      GitHub Release Information

    Travis CI creates a GitHub release in the application repository bluemix-python-app-sample and uploads the release artifact upon a successful build.

  7. Fork the deployment repository in GitHub

    1. Fork IBMCloudDevOps/bluemix-python-deploy-sample. This will be our deployment repository. 
      GitHub Fork Deploy Repository

    The separation of application and deployment described in step 2 is again reflected in our repository separation. This forked repository contains the deployment source, consisting of an Ansible Playbook (playbook.yml) which uses the CloudFoundry command line interface (CLI) tool to publish the application to Bluemix. The scripts to install Ansible, the CloudFoundry CLI, as well as those for downloading the application artifact and running the Ansible playbook are found in the bin folder of this project.

  8. Setup Travis CI to build your deployment repo

    1. Sign into Travis CI with your GitHub ID.
    2. Click the + next to My Repositories to enable Travis CI to build the new repository.
    3. Flick the repository switch on for your repository. Sync account if the repository does not appear in your list.
    4. Click the gear icon beside your repository to ‚ÄúGo to Travis CI repository settings‚ÄĚ.
    5. Set the following Travis CI environment variables (ensure Display value in build log is off to protect your information):
      • GITHUB_OAUTH_TOKEN¬†<The token we generated in step 1. This will prevent us from getting rate limited when doing API calls>
      • APP_REPO_OWNER <The owner of the app repo in GitHub you wish to deploy>
      • APP_REPO_NAME¬†bluemix-python-app-sample
      • BLUEMIX_USER <Your IBM Bluemix ID>
      • BLUEMIX_PASS¬†<Your IBM Bluemix password>
      • BLUEMIX_ORGANIZATION¬†<Your IBM Bluemix organization>
      • BLUEMIX_SPACE <Your IBM Bluemix space>
        Travis Deploy Environmental Variables_
  9. Update the deploy log

    The branch triggerDeploy creates a Deployment.log file. Update this file with your deploy information. The format provided in the file looks like:

    <YYYY-MM-DD HH:MM> <VERSION> <NOTES>

    Keeping a history of your deploys in this fashion allows you to know what is version is currently released, and when.
    If you’re team is spread across multiple time zones, you may wish to enforce a standard time zone in here.

     

  10. Run the Deployment Repo Pipeline

    1. Create a new pull request from the branch triggerDeploy to the master branch of your base repository. Make sure you are not pulling against the upstream master. 
      GitHub Deploy Repository Pull Request Branches
    2. Wait for the checks to pass and merge the pull request
  11. See the deployment repository's Travis CI build

    1. Go to your repo in Travis CI and watch the build for the master branch run. This will deploy your artifacts as a GitHub Release. This process will spawn a new Travis CI build which can be ignored.
      Travis CI Deployment Results
    2. Once Travis CI has completed, navigate to https://<APP_HOSTNAME>.mybluemix.net/. The following page will be displayed:
      Bluemix Running Python Application

    This build in¬†deploys the sample¬†application to Bluemix‘s CloudFoundry¬†by first downloading the latest release from the application respository, then using the Ansible¬†Playbook to authenticate and push the application to Bluemix.

     

  12. View the application in the Bluemix dashboard

    1. Navigate to bluemix.net and use the Log In button on the top right to authenticate
    2. On the Apps Dashboard you should see your deployed app. If you do not, ensure you are in the right Account/Region/Organization/Space from the account switcher menu at the top right of the Bluemix dashboard. 
      Bluemix App Dashboard
  13. Explanation of Process

    This recipe describes a way to manage the separation of what you application does from how its deployed. GitHub repositories are used to manage both source code and application artifacts. Travis CI provides continuous integration of your application source, as well as continuous deployment of your application using your Ansible Playbook. Updates to the master branch of the application repository (bluemix-python-app-sample) trigger the build and test of the application, and the subsequent publishing of the application artifact upon success. Updates to the Deployment.log file on the master branch of the deployment logic repository (bluemix-python-deploy-sample) trigger the Travis CI build to run the Ansible Playbook. The Ansible Playbook downloads the application artifact and delivers it to your Bluemix environment.

    Flow Overview

     

    All of the 3rd party logos above are property of their respective owners, and have been used in compliance with any guidelines set by those entities. Please see GitHub Logo Information, Travis CI Logo Information, Tar GZ Icon Information (Wikimedia Commons) for more information.

  14. Closing Thoughts

    GitHub and Travis CI are flexible tools that can enable you not only to source control your application’s source code, but also your deployment logic. Using the method described in this recipe allows for agile development and traditional releases with deployments to the cloud. Teams running development and operations separately will find such a system more useful. Try using GitHub to provide this roll separation for you today.

     

    Authors

    Jesse Antoszyk is a Software Engineer with IBM Cloud.

    Adam King is a Senior Software Engineer with IBM Cloud.

2 comments on"Separating Continuous Integration from Continuous Deployment using GitHub and Travis CI"

  1. Great work folks! Thanks for sharing this information

  2. Hey great tutorial on making TravisCI and Ansible work together. I’m getting the following error in ansibleSetup.sh script :
    The following information may help to resolve the situation:
    “`
    The following packages have unmet dependencies:

    ansible : Depends: python-cryptography but it is not installable

    E: Unable to correct problems, you have held broken packages.
    “`
    I tried force installing ansible and dependencies but to no success. Wondering if you’d faced this issue ?

Join The Discussion