Autoscale your VMware clusters on IBM Cloud

Introduction

When the workload on a cluster increases and causes the cluster to be at risk of running out of computational resources like CPU and memory, you need to scale the cluster. Adding more hosts into the cluster is one of the scaling policies known as scaling out. The new hosts bring additional amounts of CPU and memory into the overall CPU and memory at the cluster level. When the workload decreases and resources are idle or no longer used, you can remove a few hosts from the cluster in order to save the cost from paying for the rented or purchased hosts. This is known as scaling in.

An important aspect of autoscaling is automating the process of adding and removing hosts from a cluster. IBM Cloud for VMware Solutions is an automation platform for the deployment and the day 2 operations of vCenter servers and clusters, which supports adding and removing hosts. IBM Cloud for VMware Solutions also provides a REST API that automates processes that are integrated with other services. The advantage of using the IBM Cloud for VMware Solutions API is that the scaling process becomes simple and flexible because of the following:

  • IBM Cloud for VMware Solutions automation covers the complete process for adding a host, from provisioning the bare metal hosts and configuring the hosts to moving the hosts into the cluster, as well as the cluster cleanup in the process of removing the host.
  • IBM Cloud for VMware Solutions automation uses best practices in the cluster configuration. For example, all the clusters have the Distributed Resource Scheduler (DRS) enabled by default; therefore, when a host is added into a cluster or removed from a cluster, the workloads on the cluster are automatically spread across all the existing hosts on the cluster for the resource rebalance.
  • IBM Cloud for VMware Solutions REST API supports the customization of adding a host. For example, not only can you specify the number of hosts to be added, but you can also specify the hosts to be the same or different from the existing hosts in the cluster.

In some cases, the request for scaling a cluster is predictable. For example, considering the usage of CPU and memory at the cluster level is one of the key factors for determining whether the time of a scale-out or a scale-in is coming. Setting the proper thresholds and using the rules can leave the scaling actions fired without the manual intervention. In another example, the holiday seasons have seen the big rise and fall of the workload from some special applications like e-commerce; therefore, it is reasonable to plan and schedule the scaling actions in advance.

IBM Cloud Functions is a Functions-as-a-Service (FaaS) platform that provides a set of serverless apps that you can bind together and integrate with custom code to build event-driven applications. You can leverage IBM Cloud Functions to define the triggers for the scaling scenarios described above and bind the code of calling the IBM Cloud for VMware Solutions REST API to the triggers. With the automation behind the REST API, the end-to-end autoscaling solution from the triggers to the actions to the processes is built up.

This tutorial shows how you can use all of these IBM services to implement the two scenarios of autoscaling clusters: cluster resource usage-based scaling and time-based scaling. The following chart describes how IBM Cloud for VMware Solutions, IBM Cloud Functions, and other IBM services work together:

Autoscaling solution on IBM Cloud

With IBM Cloud Identity and Access Management (IAM), all IBM services used here can share a single IAM key under one IBM Cloud account. This helps save the work of managing the accesses of different services.

Learning objectives

  • Understand the IBM Cloud for VMware Solutions REST API and usage
  • Use the VMware PowerCLI to collect the statistical data of the vCenter cluster
  • Use IBM Cloud Functions to integrate the IBM Cloud for VMware Solutions REST API

Prerequisites

  • An IBM Cloud account(?cmsp=ibmdev--developer-tutorials-_-cloudreg) and a valid IAM key
  • A dedicated vCenter Server instance on IBM Cloud for VMware Solutions and a machine with the VMware PowerCLI installed connected to the vCenter Server
  • An IAM-enabled Cloudant instance and an IAM-enabled Functions namespace on IBM Cloud
  • IBM Cloud CLI

Estimated time

Completing this tutorial should take about 1 hour.

Implement cluster resource usage-based scaling

This section gives the instructions for cluster resource usage-based scaling.

Create cluster resource usage collector

The collector is written in PowerShell and uses VMware PowerCLI to connect to the vCenter Server. VMware PowerCLI provides the cmdlets of retrieving statistical data from various components on vCenter. Get-Stat can return the maximum, minimum, or average usage of CPU, memory, disk, network bandwidth, and more from hosts, clusters, or virtual machines on a vCenter Server in a given period of time. The collector uses Get-Stat to get the average percentage of the cluster CPU and memory usage in the last 12 hours. You can choose the different time periods according to the workload activity pattern on your clusters.

The collector uses the IBM Cloud for VMware Solution Rest API to get the vCenter credential. The usage is described in Step 2 of the Build inheritance trees for virtual machines and templates tutorial, which includes the code written in Python language.

The collector writes the collected data into the database in the Cloudant instance on IBM Cloud. If you don’t have the IBM Cloudant instance yet, go to the IBM Cloud catalog to create one. This tutorial uses a Cloudant instance named my_cloudant and a database in my_cloudant named monitoring. The instance my_cloudant is with a credential named cloudant-cred.

Each cluster has a document with the cluster name as the document ID in the monitoring database. It stores the collected CPU and memory usage percentage and also the ID of the vCenter instance. Here is an example of the document for the cluster named workload-cluster:

{
    "_id": "workload-cluster",
    "_rev": "76-3f4c7a4df17797ad5467de53d0ce879c",
    "instanceID": "82a265da-3949-49f5-8bb1-c5a199462156",
    "cpuUsagePercentage": 1.40
    "memoryUsagePercentage": 41.97
}

The collector script uses the Cloudant REST API to perform the Cloudant operations, such as read and update.

The complete content of the script is shown here:

# file name: collector.ps1

param(
    [string]$ic4vURI,
    [string]$iamApiKey,
    [string]$cloudantURI,
    [string]$vCenterInstanceID
)

function authenticate([String] $apiKey){
    $body = @{
        "grant_type"="urn:ibm:params:oauth:grant-type:apikey"
        "apikey"=$apiKey
    }
    $header = @{
        "Accept"="application/json"
    }
    $params = @{
        Uri         = 'https://iam.cloud.ibm.com/identity/token'
        Headers     = $header
        Method      = 'POST'
        Body        = $body
        ContentType = 'application/x-www-form-urlencoded'
    }
    $response = Invoke-RestMethod @params
    return $response.access_token
}

function GetVcenterCredentials([String] $instanceID){

    $accessToken = authenticate $iamApiKey
    $vcenterInstanceURI = $ic4vURI + "/v1/vcenters/" + $instanceID
    $header = @{
        "Content-type"="application/json"
        "Authorization"="Bearer " + $accessToken
    }
    $vcenter = Invoke-RestMethod -Uri $vcenterInstanceURI -Method "GET" -Headers $header
    $return = @()
    foreach ($endpoint in $vcenter.endpoints) {
        if ($endpoint.type -eq "vCenter/PSC") {
            foreach( $credential in $endpoint.credentials){
                if ($credential.user -eq "Administrator@vsphere.local"){
                    $return += @{
                        "ipAddress"=[String]$endpoint.ip_address.Trim()
                        "user"=[String]$credential.user.Trim()
                        "password"=[String]$credential.password.Trim()
                    }
                    return $return
                }
            }
        }
    }
}

function CollectClusterResourceUsage([Int] $hours) {
    $start = (Get-Date).AddHours(- $hours)
    $stat = "cpu.usage.average","mem.usage.average"
    $clusterResourceUsage = foreach ($cluster in Get-Cluster) {
        Get-Stat -Entity $cluster -Start $start -Stat $stat | Group-Object -Property {$_.Entity.Name} |
        select  @{N="Name";E={$cluster.Name}},
                @{N="MEMAvg";E={$_.Group | Where {$_.MetricId -eq "mem.usage.average"} |
                    Measure-Object -Property Value -Average | Select -ExpandProperty Average}},
                @{N="CPUAvg";E={$_.Group | Where {$_.MetricId -eq "cpu.usage.average" -and $_.Instance -eq ""} |
                    Measure-Object -Property Value -Average | Select -ExpandProperty Average}}
    }
    return $clusterResourceUsage
}

function updateDatabase([String] $clusterName, [double] $cpuUsagePercentage, [double] $memoryUsagePercentage){
    $accessToken = authenticate $iamApiKey
    $cloudantDocURI = $cloudantURI + "/monitoring/" + $clusterName
    $header = @{
        "Content-type"="application/json"
        "Authorization"="Bearer " + $accessToken
    }

    $data = Invoke-RestMethod -Uri $cloudantDocURI -Method "GET" -Headers $header
    $newData = @{
        "_id" = $data._id
        "_rev" = $data._rev
        "instanceID" = $vCenterInstanceID
        "cpuUsagePercentage" = $cpuUsagePercentage
        "memoryUsagePercentage" = $memoryUsagePercentage
    }
    $newDataJson = $newData|ConvertTo-Json
    Invoke-RestMethod -Uri $cloudantDocURI -Method "PUT" -Headers $header -Body $newDataJson
}

$vcenter = GetVcenterCredentials  $vCenterInstanceID

Connect-VIServer -Server $vcenter.ipAddress  -Protocol https -User $vcenter.user -Password $vcenter.password

$usageClusters = CollectClusterResourceUsage 12

ForEach ($c in $usageClusters) {
    updateDatabase $c.Name $c.CPUAvg $c.MEMAvg
}

The script collector.ps1 runs on the machine with the connectivity to your vCenter server, and you need to execute it periodically to keep the collected data updated in the Cloudant database. To do that, you can use a Cron job or a scheduled task, which depends on the operation system of your machine. This tutorial uses a Linux machine to run the collector; therefore, the script is executed by a Cron job defined here in crontab:

crontab -l
*/30 * * * * pwsh /root/collector.ps1 <Your IAM API KEY> <Your Cloudant instance URI> <Your vCenter instance ID>

With this Cron job, the collector.ps1 is executed every 30 minutes.

The rest of the steps show how to use IBM Cloud Functions to trigger the scaling actions based on the changes of the cluster CPU and memory usage from the Cloudant database. This chart describes the components and their relation to the concept of IBM Cloud Functions.

Cluster resource usage-based scaling

The scaling task is performed with three actions in a sequence named UsageBasedScalingSequence. The action of read manages to connect to the Cloudant instance and retrieve the content of all cluster documents from the Cloudant database, which contain the statistical data of CPU and memory usage. The read action passes the data to the next action, DecisionAction. The DecisionAction has the predefined logic to determine if a scaling action is needed for the cluster by inspecting current statistic data; it makes the decision and passes the decision to the last action ScalingAction. The ScalingAction adds hosts, removes hosts, or does nothing based on the decision.

One public action is used for the read action because it already contains the code snippet of fetching the documents from the specified Cloudant database. The DecisionAction and ScalingAction are customer actions, and they are provided with the code of the decision-making logic and the code of calling the IBM Cloud for VMware Solutions REST API.

The UsageBasedScalingTrigger is a trigger binding to the Cloudant database monitoring. Every change to the database fires the trigger. The rule UsageBasedScalingRule associates UsageBasedScalingTrigger to the sequence UsageBasedScalingSequence such that the actions in the sequence are invoked when the trigger is fired.

The IBM Cloud CLI with the Cloud Functions CLI plug-in (for example, commands ibmcloud fn) is used in this tutorial to create the IBM Cloud Functions triggers, actions, and rules, as well as mashing them up into the structure shown in the previous chart.

All the ibmcloud fn commands used in the following code snippets run in an IAM enabled Function namespace. If you don’t have a Function namespace yet, go to the Managing namespaces page to learn how to create one. Run the command ibmcloud fn property set --namespace <namespace id> to switch to the namespace that you created before you run the other ibmcloud fn commands.

Create the package binding to your IBM Cloudant database

You need to create a Cloudant type trigger with the feed from a Cloudant package binding to your IBM Cloudant database.

Run the following command to create a /whisk.system/cloudant package binding named scaling-cloudant-binding.

$ ibmcloud fn package bind /whisk.system/cloudant scaling-cloudant-binding

Run the following command to bind the service cloudantnosqldb to the binding package scaling-cloudant-binding with the Cloudant instance name and the credential name provided.

$ ibmcloud fn service bind cloudantnosqldb scaling-cloudant-binding --instance my-cloudant --keyname 'cloudant-cred'

If you don’t know the name of your Cloudant credentials, run the command ibmcloud resource service-keys --instance-name <cloudant instance name> to find them.

Create cloudant trigger

Run the following command to create a trigger named UsageBasedScalingTrigger with the changes feed from the package binding created in the previous step and with the Cloudant database monitoring as `dbname` parameter provided.

$ ibmcloud fn trigger create UsageBasedScalingTrigger --feed /_/scaling-cloudant-binding/changes --param dbname monitoring

Create custom action

When you are making the decision to scale, consider the following information:

  • Thresholds of CPU and memory usage percentage: This is the basic scaling criteria in the cluster resource usage-based scaling scenario. By comparing the current CPU and memory usage percentage with the predefined scale-out and scale-in thresholds of CPU and memory usage percentage, you can decide whether it is the time to scale out or scale in the cluster for more CPU and memory resource or withdraw some.

  • Status of the cluster: When the IBM Cloud for VMware Solutions API of adding or removing hosts is invoked, it requests to start the automation process instead of actually running through the automation process. The process could take up to one hour to complete for adding hosts and half an hour for removing hosts. While the process is running, you should not make another scaling decision because it isn’t going to be accurate while the hosts are in the process of being added or removed. The status of the cluster is needed for determining if the scaling process is completed or not. In IBM Cloud for VMware Solutions, when an automation process of adding hosts or removing hosts starts, the cluster status changes from ReadyToUse to Modifying, and the status does not change back to ReadyToUse until the process is complete. This can help determine if another scaling should occur or not at the current time. In particular, if the cluster is not at ReadyToUse, no scaling action will be concluded even though the current CPU and memory usage percentage is already beyond the threshold.

  • Maximum and minimum size of the cluster: This considers the limit of the host number, which can be supported in VMware clusters as well as the budget planned on a workload cluster. If the cluster continues to scale out or scale in, it will either reach the host number limit and cause the failure of adding or removing hosts, or it will make the cluster exceed the planned budget. You can avoid both of these by checking the allowed maximum or minimum host number of the cluster in the scaling decision.

With all the above facts considered, the following decision tree is put in the DecisionAction.

Decision Tree

Run the following command to create an action named DecisionAction with the Python script named scaling_decision.py provided and with a group of parameters including the thresholds of CPU and memory usage percentage, and the allowed maximum and minimum host number in the cluster.

$ ibmcloud fn action create DecisionAction scaling_decision.py --param scaleout_threshold_cpu 80 --param scaleout_threshold_memory 80 --param scalein_threshold_cpu 10 --param scalein_threshold_memory 10 --param max_allowed_host_num 32 --param min_allowed_host_num 2

The initial threshold of CPU and memory usage percentage for adding hosts is set to 80%, and the threshold for removing hosts is set to 10%. The allowed cluster size is set between 2 and 32 hosts. You can adjust all of those parameters later by running the fn action update commands, as shown below.

$ ibmcloud fn action update ResourceUsageBasedAction --param scaleout_threshold_cpu <new value> --param scaleout_threshold_memory <new value> --param scalein_threshold_cpu <new value> --param scalein_threshold_memory <new value>
$ ibmcloud fn action update ResourceUsageBasedAction --param max_allowed_host_num <new value> --param min_allowed_host_num <new value>

The Python script scaling_decision.py implements the decision tree. The entry function main accepts the JSON-formatted parameter including the content of the cluster resource usage document from the Cloudant database, as well as the values of the thresholds and the allowed max and min cluster size. It returns the decision result in the JSON format for the next scaling action to use. The IBM Cloud for VMware Solutions API is used here to get the cluster status and the number of the hosts in the cluster.

# file name: scaling_decision.py

import os
import requests

ic4v_base_uri = 'https://api.vmware-solutions.cloud.ibm.com'
auth_uri = 'https://iam.cloud.ibm.com/identity/token'


def authenticate():
    api_key = os.environ['__OW_IAM_NAMESPACE_API_KEY']
    headers = {'Accept': 'application/json'}
    params = {
        'grant_type': 'urn:ibm:params:oauth:grant-type:apikey',
        'apikey': api_key
    }
    resp = requests.post(auth_uri, data=params, headers=headers)
    if resp.status_code == 200:
        json = resp.json()
        headers['Authorization'] = 'Bearer %s' % json['access_token']
        headers['X-Auth-Refresh-Token'] = json['refresh_token']
        return headers
    else:
        raise Exception('Failed to get token')


def get_cluster_info(instance_id, cluster_name):
    uri = '%s/v1/vcenters/%s/clusters' % (ic4v_base_uri, instance_id)
    headers = authenticate()
    resp = requests.get(uri, headers=headers)
    clusters = resp.json() if resp.status_code == 200 else []
    for cluster in clusters:
        if cluster['name'] == cluster_name:
            uri = '%s/v1/vcenters/%s/clusters/%s' % (ic4v_base_uri, instance_id, cluster['id'])
            cluster_resp = requests.get(uri, headers=headers)
            return cluster_resp.json() if resp.status_code == 200 else {}


def main(params):
    cluster_name = params['_id']
    instance_id = params['instanceID']
    cluster_info = get_cluster_info(instance_id, cluster_name)

    decision = {
        "cluster_name": cluster_name,
        "instance_id": instance_id,
        "host_num": 1,
        "scaling_type": None
    }

    if cluster_info['status'] == 'ReadyToUse':
        hosts = cluster_info.get('hosts', [])

        if params['cpuUsagePercentage'] > params['scaleout_threshold_cpu']\
                or params['memoryUsagePercentage'] > params['scaleout_threshold_memory']:
            if len(hosts) < params['max_allowed_host_num']:
                decision['scaling_type'] = 'scale-out'
            else:
                print('The current hosts already reaches the allowed maximum number')

        if params['cpuUsagePercentage'] < params['scalein_threshold_cpu']\
                and params['memoryUsagePercentage'] < params['scalein_threshold_memory']:
            if len(hosts) > params['min_allowed_host_num']:
                decision['scaling_type'] = 'scale-in'
            else:
                print('The current hosts already reaches the allowed minimum number')
    else:
        print('Scaling is not allowed under current cluster status: %s' % cluster_info['status'])

    print('Scaling decision is %s' % decision['scaling_type'])
    return decision

Create a custom action

Run the following command to create the action named ScalingAction with the Python script scaling.py provided.

$ ibmcloud fn action create ScalingAction scaling.py

The script scaling.pyimplements calling the IBM Cloud for VMware Solutions API to add hosts to or remove hosts from the given cluster.

The entry function main accepts the JSON-formatted parameter including the ID of the vCenter instance, the name of the cluster to scale, and the scaling type. The scaling type decides the different code paths for calling the API for adding hosts or calling the API for removing hosts. The parameter also provides the number of the hosts to add or remove. For the add-host API call, the payload is created to include the host number; for the remove-host API call, the payload is created to include the ID list of the hosts to be deleted, which are chosen from the latest added hosts.

# file name: scaling.py

import os
import requests
import json

ic4v_base_uri = 'https://api.vmware-solutions.cloud.ibm.com'
auth_uri = 'https://iam.cloud.ibm.com/identity/token'


def authenticate():
    api_key = os.environ['__OW_IAM_NAMESPACE_API_KEY']
    headers = {'Accept': 'application/json'}
    params = {
        'grant_type': 'urn:ibm:params:oauth:grant-type:apikey',
        'apikey': api_key
    }
    resp = requests.post(auth_uri, data=params, headers=headers)
    if resp.status_code == 200:
        json = resp.json()
        headers['Authorization'] = 'Bearer %s' % json['access_token']
        headers['X-Auth-Refresh-Token'] = json['refresh_token']
        return headers
    else:
        raise Exception('Failed to get token')


def get_cluster_info(instance_id, cluster_name):
    uri = '%s/v1/vcenters/%s/clusters' % (ic4v_base_uri, instance_id)
    headers = authenticate()
    resp = requests.get(uri, headers=headers)
    clusters = resp.json() if resp.status_code == 200 else []
    for cluster in clusters:
        if cluster['name'] == cluster_name:
            uri = '%s/v1/vcenters/%s/clusters/%s' % (ic4v_base_uri, instance_id, cluster['id'])
            cluster_resp = requests.get(uri, headers=headers)
            return cluster_resp.json() if resp.status_code == 200 else {}
    return {}


def add_host(instance_id, cluster_name, host_num):
    cluster_info = get_cluster_info(instance_id, cluster_name)

    header = authenticate()
    header['Content-Type'] = 'application/json'
    payload = {
        "quantity": host_num
    }
    uri = '%s/v1/vcenters/%s/clusters/%s/hosts' % (ic4v_base_uri, instance_id, cluster_info['id'])
    resp = requests.post(uri, headers=header, data=json.dumps(payload))
    return resp.status_code == 202, resp.content


def remove_host(instance_id, cluster_name, host_num):
    cluster_info = get_cluster_info(instance_id, cluster_name)
    hosts = cluster_info.get('hosts', [])
    hosts = sorted(hosts, key=lambda i: i['provision_date'])

    header = authenticate()
    header['Content-Type'] = 'application/json'
    playload = {
        "action": "delete",
        "hosts": [h['id'] for h in hosts[:host_num]]
    }
    uri = '%s/v1/vcenters/%s/clusters/%s/hosts' % (ic4v_base_uri, instance_id, cluster_info['id'])
    resp = requests.patch(uri, headers=header, data=playload)
    return resp.status_code == 202, resp.json()


def main(param):
    if param['scaling_type'] == 'scale-out':
        result, msg = add_host(param['instance_id'], param['cluster_name'], param['host_num'])
    elif param['scaling_type'] == 'scale-in':
        result, msg = remove_host(param['instance_id'], param['cluster_name'], param['host_num'])
    else:
        result = False
        msg = 'The scaling action is not supported'

    if result:
        return {"message": "succeeded"}
    else:
        return {"message": msg}

Create action sequence

The sequence to create here includes the pubic read action from /whisk.system/cloudant package and the two custom actions, DecisionAction and ScalingAction, that you created.

Run the following command to create the sequence named UsageBasedScalingSequence and add read, DecisionAction, and ScalingAction actions.

$ ibmcloud fn action create ResourceBasedScalingActionSequence --sequence <full path of read action in scaling-cloudant-binding package>, <full path of DecisionAction>, <full path of ScalingAction>

Run the following command to get the full path of the read action from the binding package scaling-cloudant-binding.

$ ibmcloud fn package get <full path of scaling-cloudant-binding>

If you don’t know the full path of the binding package, run ibmcloud fn package list to query first.

Run ibmcloud fn action list to get the full path of the DecisionAction and ScalingAction actions.

Create a rule

Run the following command to create the rule named UsageBasedScalingRule to associate the trigger UsageBasedScalingTrigger and the sequence UsageBasedScalingSequence.

$ ibmcloud fn rule create UsageBasedScalingRule UsageBasedScalingTrigger UsageBasedScalingSequence

The IBM Cloud for VMware Solutions API used in this section includes:

  • POST /v1/vcenters/{instance_id}/clusters/{cluster_id}/hosts: Add hosts to a specific cluster.
  • PATCH /v1/vcenters/{instance_id}/clusters/{cluster_id}/hosts: Remove hosts from a specific cluster.
  • GET /v1/vcenters/{instance_id}/clusters/{cluster_id}: Get the cluster details, including it hosts.
  • GET /v2/vcenters/{instance_id}/clusters: List all clusters with the cluster name and cluster ID.

Implement time-based scaling

This section gives the instructions for cluster time-based scaling.

In the time-based scaling scenario, the scaling action is performed on a specified date. This chart describes the triggers and the action used in the scenario.

Time-based scaling

Two triggers, TimeBasedScaleoutTrigger and TimeBasedScaleinTrigger, are bound to the Alarms service with the different dates for the expected scale-out and scale-in. On the scheduled dates, the triggers are fired to invoke the ScalingAction action, which is associated with the triggers by the rules TimeBasedScaleoutRule and TimeBasedScaleinRule.

Create alarm triggers

The Alarms type triggers used in this tutorial only fire once on the scheduled times; therefore, the triggers are created with the feed of /whisk.system/alarms/once.

Run the following two commands to create two triggers (TimeBasedScaleoutTrigger and TimeBasedScaleinTrigger) of type Alarm with the different dates and scaling types.

$ ibmcloud fn trigger create TimeBasedScaleoutTrigger --feed /whisk.system/alarms/once --param trigger_payload "{\"scaling_type\":\"scale-out\", \"host_num\": 2, \"cluster_name\":\"workload-cluster\"}" --param date "2020-12-24T00:00:00.000Z"
$ ibmcloud fn trigger create TimeBasedScaleinTrigger --feed /whisk.system/alarms/once --param trigger_payload "{\"scaling_type\":\"scale-in\", \"host_num\": 2, \"cluster_name\":\"workload-cluster\"}" --param date "2021-1-2T00:00:00.000Z"

The TimeBasedScaleoutTrigger is for adding 2 hosts and is fired at the beginning of the day on 24 December 2020; the TimeBasedcaleinTrigger is for removing 2 hosts and is fired at the end of the day of 01 January 2021.

Create rules

The TimeBasedScaleoutTrigger and TimeBasedScaleinTrigger share the same ScalingAction created in the steps for implementing the cluster resource usage-based scaling scenario. The script in ScalingAction goes to the different paths of adding hosts and removing hosts based on the scaling_type in trigger_payload defined in the triggers.

Run the following two commands to create the rules named TimeBasedScaleoutRule and TimeBasedScaleinRule and associate the TimeBasedScaleoutTrigger and TimeBasedScaleinRule triggers with ScalingAction.

$ ibmcloud fn rule create TimeBasedScaleoutRule TimeBasedScaleoutTrigger ScalingAction
$ ibmcloud fn rule create TimeBasedScaleinRule TimeBasedScaleinRule ScalingAction

Monitor scaling activities

You can use the IBM Cloud Functions dashboard to monitor the activities of the created triggers and actions and track the occurrences of the scaling actions to your clusters.

Monitor scaling activities

You can find the output of the script in each action showing up in Activity Log to know the result of scaling decisions and scaling actions.

Here is an example from one invocation of the DecisionAction action.

{
    "activationId": "7d920648ba904f30920648ba902f30b4",
    "annotations": [{
        "key": "causedBy",
        "value": "sequence"
    }, {
        "key": "path",
        "value": "7c3ffaa6-e6b2-45ea-8337-9ba8ee90aeb0/DecisionAction"
    }, {
        "key": "kind",
        "value": "python:3.7"
    }, {
        "key": "timeout",
        "value": false
    }, {
        "key": "limits",
        "value": {
            "concurrency": 1,
            "logs": 10,
            "memory": 256,
            "timeout": 60000
        }
    }],
    "cause": "b0bace6795384378bace67953883788b",
    "duration": 232,
    "end": 1593918009531,
    "logs": ["2020-07-04T11:30:03.467102Z    stdout: Scaling decision is scale-out"],
    "name": "DecisionAction",
    "namespace": "7c3ffaa6-e6b2-45ea-8337-9ba8ee90aeb0",
    "publish": false,
    "response": {
        "result": {
            "instance_id": "82a265da-3949-49f5-8bb1-c5a199462156",
            "scaling_type": "scale-out",
            "host_num": 1
        },
        "status": "success",
        "success": true
    },
    "start": 1593918009299,
    "subject": "ServiceId-ef2edd10-c0d5-415c-a01c-2bac16832d61",
    "version": "0.0.1"
}

Summary

After completing this tutorial, you should know the IBM Cloud for VMware Solutions REST API for adding hosts to and removing hosts from a cluster on the vCenter instance on IBM Cloud for VMware Solutions platform. You should also be able to use IBM Cloud Functions to create the triggers and actions for the two scenarios of cluster autoscaling: cluster resource usage-based scaling and time-based scaling.