Digital Developer Conference: Hybrid Cloud 2021. On Sep 21, gain free hybrid cloud skills from experts and partners. Register now

Automate GitHub Enterprise with IBM Cloud Functions

Introduction

When you are the administrator of a GitHub repository, you can automate many of the tasks, including assigning issues, adding labels, opening pull requests, and moving issues through agile processes like Kanban.

You can perform this automation on IBM Cloud with IBM Cloud Functions, a serverless Function-as-a-Service (FaaS) platform that is based on Apache OpenWhisk. Cloud Functions uses highly scalable, performant functions (written in almost any language) while minimizing compute costs.

If your repository is hosted in a public GitHub account, everything works fine, but unfortunately, I found that the default system package named /whisk.system/github does not work on GitHub Enterprise repositories.

In this article, I walk you through a use case to show you the workaround I found for this limitation.

The use case

In the use case for this article, we want to label GitHub issues with the month name whenever they are created in order to quickly filter them according to their opening date. This is a boring administration task that is ideal for automation.

As the Cloud Functions documentation details, you can use the platform to create stateless code snippets that are set to perform specific tasks. These pieces of code, called actions, can be invoked (executed) whenever an associated event triggers them.

In the Cloud Functions/OpenWhisk programming model, event sources (such as a GitHub channel) send their events through feed services, which adapt the event data and fire compatible triggers that are associated with one or more actions by using rules.

The sequence is:

  1. An event happens.
  2. Triggers are fired.
  3. Actions are invoked to automate tasks.

This reactive pattern is exactly the pattern we need for our use case. In terms of the model, the event is the GitHub issue being created, thanks to the webhook.

Using the GitHub API, it is easy to write the code snippet (as shown below) to create an action in Cloud Functions named label_add:

javascript
def main(args):
    number = args['issue']['number']
    token = args['accessToken']
    repository = args['repository']
    month = datetime.datetime.now().strftime("%B")
    session = requests.session()
    session.headers = {'Authorization': f'token {token}','Content-Type': 'application/json'}
    url = f'https://github.ibm.com/api/v3/repos/{repository}/issues/{number}/labels'
    response = session.post(url=url, json={'labels': [month]})
    return {'response': json.loads(response.content.decode('utf-8'))}

Once you create the action, you can declare a trigger to react to GitHub issue creation, and then you can write the rule to associate the action and trigger. You can now browse Cloud Functions built-in packages and find the /whisk.system/github package, which provides you with the feed you need. You then configure this with your GitHub repo location and credentials.

You are done! Well, almost done.

When I tested my version of /whisk.system/github package, I saw that it does not work with Enterprise GitHub repositories.

So, how can we solve this limitation?

The simple (polite) solution

Wait a moment…

The package that provides the GitHub feed service in /whisk.system is open source. Let’s examine the package source code of my version to understand the reason we hit the GitHub Enterprise limitation. Looking at the source code, you can see that the GitHub API endpoint is hardcoded to the one used for the public GitHub (in other words, api.github.com/repos/):

javascript
 // The URL to create the webhook on Github
  var registrationEndpoint = 'https://api.github.com/repos/' + (organization ? organization : username) + '/' + repository + '/hooks';
  console.log("Using endpoint: " + registrationEndpoint);

Having the package code for the GitHub webhook, you can copy it locally and change the API endpoint used to the one for your GitHub Enterprise account: Simply replace the hardcoded value with your enterprise endpoint. You can find the GitHub Enterprise URL by following this GitHub documentation.

The API endpoint has the form https://[hostname]/api/v3; in our case, it is https://github.ibm.com/api/v3.

The modified package code then looks like the following:

javascript
 // The URL to create the webhook on GitHub
  var registrationEndpoint = 'https://github.ibm.com/api/v3/repos/' + (organization ? organization : username) + '/' + repository + '/hooks';
  console.log("Using endpoint: " + registrationEndpoint);

At this point, you can try to create a new feed in Cloud Functions using the modified source by creating it as a new action and setting the feed attribute to true. Using the IBM Cloud CLI and the functions plug-in, the command looks like the following:

shell
ibmcloud functions action create new_feed fsthook.js -a feed true

Then, you can also incorporate the GitHub parameters that the action needs to work and update the feed action:

shell
ibmcloud functions action update new_feed fsthook.js -a feed true -p username Fausto-Ribechini -p accessToken b02xxxxxxxxxxxxxxxxxxxxx1a1

Now, you really can create a trigger using the new modified feed referring to the repository and event you are interested in:

shell
ibmcloud functions trigger create mytrigger --feed new_feed --param events issues --param repository fstorg/test_reviews

After the command of trigger creation is issued, you receive an output showing what is normally recorded in the actvationid log. There, you also find the response for creating the integration to GitHub:

"response": {
        "result": {
            "response": {
                "active": true,
                "config": {
                    "content_type": "json",
                    "insecure_ssl": "0",
                    "url": "https://466ae588-acbd-41aa-8901-7148bb974675:BWXxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxGN3@eu-gb.functions.cloud.ibm.com/api/v1/namespaces/_/triggers/mytrigger"
                },
                "created_at": "2021-05-04T15:13:49Z",
                "events": [
                    "issues"
                ],
                "id": 9367385,
                "last_response": {
                    "code": null,
                    "message": null,
                    "status": "unused"
                },
                "name": "web",
                "ping_url": "https://github.ibm.com/api/v3/repos/fstorg/test_reviews/hooks/9367385/pings",
                "test_url": "https://github.ibm.com/api/v3/repos/fstorg/test_reviews/hooks/9367385/test",
                "type": "Repository",
                "updated_at": "2021-05-04T15:13:49Z",
                "url": "https://github.ibm.com/api/v3/repos/fstorg/test_reviews/hooks/9367385"
            }
        },
        "size": 708,
        "status": "success",
        "success": true
    },

You have your webhook ready and configured. You can confirm this by going to your repository’s GitHub settings and viewing its Hooks section.

hooks.png

Digging deeper

By creating the new feed action with proper parameter values in Cloud Functions, it has automatically registered a new webhook in your respective GitHub Enterprise account for the repository. This causes GitHub to send messages (events) in JSON format with “payload” data describing the changes made (in our use case, for any issue-related operations).

Click the newly added webhook to open its configuration and view the webhook management settings.

editing_hook.png

The Content type field shows that the HTTP payload information sent to any webhook will be in JSON format (in other words, have the HTTP header “content-type” key set to “application/json”).

Note: You should always set GitHub feed (webhook) payloads to application/json when working with Cloud Functions.

Look at the Payload URL field and note the URL. There was a similar URL in the response provided during the trigger creation shown previously:

json
"url": "https://466ae588-acbd-41aa-8901-7148bb974675:BWXxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxGN3@eu-gb.functions.cloud.ibm.com/api/v1/namespaces/_/triggers/mytrigger"

This URL is formed in the following way and is prepended using your Cloud Functions account’s basic authentication token as whisk_auth:

shell
https://{whisk auth}@{whisk API host}/api/v1/namespaces/{whisk namespace}/triggers/{trigger_name}

The whisk_auth, whisk API host and whisk_namespace fields are properties that you can retrieve by entering the following command:

shell
ibmcloud functions property get

An even simpler way to provide the whisk_namespace part of the URL is to use the underscore character (“_”), which tells Cloud Functions to use the account’s default Cloud Foundry namespace.

Note: Currently, GitHub webhooks are only valid for Cloud Foundry namespaces that permit basic authentication tokens to be prepended. GitHub does not support any mechanism to use OAuth2 tokens for webhooks or for IBM Cloud Identity and Access Management namespaces.

This is the URL that addresses the trigger to be fired, written with the authorization fields:

https://466ae588-acbd-41aa-8901-7148bb974675:BWhXxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxGN3@eu-gb.functions.cloud.ibm.com/api/v1/namespaces/_/triggers/mytrigger

This configuration is so simple that you could manually supply a URL to your trigger without using any feed.

Fine-tuning the events types and payload

In the last section of the Manage webhook settings, you can choose which events should trigger your webhook. You can choose to receive more than one event type. For example, you can choose to be notified by specific events, such as when an issue label is added or deleted.

As you can see, you can create a GitHub webhook in this direct way and your integration between GitHub and Cloud Functions still works. However, using a feed is preferred because you can use it to parse the JSON and simplify the data for multiple actions as action parameters. Therefore, a feed logic can avoid duplicating code in every action and be more efficient.

To test this direct method, open the settings in another repository, go to its webhooks settings, and create a new webhook using the same URL. It works!

You could now avoid creating a feed altogether if you want.

The final solution

You are ready to complete the end-to-end scenario. Create a rule to connect a trigger with your original action.

shell
ibmcloud functions rule github_integration mytrigger label_add

Let’s try this second repository, the one manually configured, and see what happens.

Open an issue in the second repository and then go back to Cloud Functions to look at the activation logs from your action.

You should find that the trigger has been fired and the action ran successfully. The action results from label_add confirmed completion of the end-to-end process, and you can see that the new brightening label was added.

Everything works as expected.

Summary

With just a few simple steps in Cloud Functions, you can automate tasks so that you can monitor and act upon any change.

In this article, you learned how to pass a webhook URL to have a trigger fired on an event sent from an external service. By using this method, you avoid being dependent on a feed. This means that by using basic authentication tokens, you can wake up a trigger using a URL built with the URI path that points to it, causing it to fire one or more actions by connecting them using rules.

By understanding the underlying logic, you can use other product webhooks to send events to Cloud Functions automation, even if a specific feed does not exist. At the same time, directly managing GitHub events can help you to understand the feasibility of other integrations using code and issue data stored in your GitHub Enterprise repositories.