Throttling is a key capability for environments that link to downstream services whether  running in an ESB or microservices.

Two common cases when you need to add API throttling include:

  • Prior to the invocation of an API to help protect downstream services
  • At the start of the API to provide a total limit of requests from all consumers

A throttling limit is expressed in tokens per interval. The token represents the monitored resources. The number of tokens available in an interval is called a bucket.  For each request, a defined number of tokens in the bucket can be requested. When the available tokens in the bucket are enough for a request, the request is accepted and requested tokens are removed from the bucket. Otherwise, the threshold is exceeded. When the threshold is exceeded, the next incoming request is rejected until the next interval. API Connect supports throttling at a per-consumer level, but it is also useful to protect APIs as a whole.

Here’s an example:

  • The user wants to limit the requests to an account look-up service
  • The throttling policy would be used in the API that calls the service
  • The rate set in the throttling policy is just under the maximum throughput the account look-up service can handle

This guide will introduce a custom throttling policy that can be applied to APIs built using the assembly. Each time the API is called from any consumer, the throttling policy checks that the total number of calls has not exceeded the limit. If it has? The request is rejected. This will help protect the account look-up from being overloaded.

Throttling Policy

The Throttling Policy is a User Defined Policy.  Once it is loaded, it can be applied to any API assembly that will be deployed to a catalog.

throttling policy

The Throttling Policy takes four parameters:

Key The unique identifier that represents a defined threshold for the specific traffic type.

The key can be any characters accessible in the GatewayScript file. If a second rate limit is created with the same key, it reuses the existing rate limit and shares the threshold. This can be configured to work across multiple appliances.

Token The maximum number of resources that can be requested during an interval, or the rate-based threshold.

This parameter also indicates the number of tokens that the bucket contains when the bucket is initialized. The bucket is initialized when this API is called. This parameter is an integer. The value is in the range 0 – (253 – 1).

Interval The frequency between quota enforcements.

This parameter is an integer. The unit is second. The value is in the range 1 – 31536000.

Interval Type The type of interval.

This parameter is a string. This parameter value is ‘fixed’ or ‘rolling.’ The default value is ‘fixed’

When the throttling limit is exceeded, all future messages will be rejected until the limit resets. A 429 status code is returned to the assembly.

Under the Covers

The Throttling Policy makes use of the DataPower Quota Enforcement functionality that was added in v7.5.

Quota enforcement provides precise threshold specification for traffic control, and supports flexible counting for any specific occurrences and counting for concurrent transactions. Administrative control and possible sanctions are imposed on requests when the threshold is exceeded.

Quota enforcement is implemented by calling a GatewayScript file on the processing action, such as GatewayScript action, for a service. The GatewayScript file calls the ratelimit module that defines and manages the rate limit buckets. Based on your DataPower configuration, the rate limit buckets are persisted on the RAID volume or stored in memory.

The code inside of the Throttling Policy is based around the sample below.

var rl = require('ratelimit');
var rate = rl.rateCreate(name, number, interval, intervalType);

Once this is created you can then remove tokens as needed with the following command:

rate.remove(1, function(err, remaining, timeToReset) {});

For more information on quota enforcement, please visit the Knowledge Center.

Download and load artifacts

  1. Download the zip files from the Github release page.
  2. Import the Policy into API Connect Catalog required (see this article on the Knowledge Center for detailed instructions.)

If you find this throttling policy useful, please try it out and let us know what you think in the comments below!

Further Reading
Go full throttle: The essentials of throttling in your application architecture


2 comments on"API Level Throttling in API Connect"

  1. MaheshwarraoCh September 20, 2017

    Hi Chris,
    Can you please provide the implmentations steps ?

  2. carlosGonzalezIBM June 21, 2018

    Hello, any way to sync the remaining seconds with natural days, minutes ..etc ?

Join The Discussion

Your email address will not be published. Required fields are marked *