Within an API Connect implementation, many events occur every second and it is crucial that we can record and retain them. These events are stored within API Connect normally and can result from many things. This includes business transactions, system processes, network operations, user interactions or something else that has incurred some sort of action, interruption or change to your API Connect environment. These events range from system level, to application level, to micro-service level. Some examples are:

  • Creating a new organization within the API Manager
  • Creating an API
  • Publishing a product to a catalog
  • Creating a new user in the developer portal
  • Creating a Multi-protocol GW policy in the DataPower gateway
  • Deleting an application
  • Shutting down a machine
  • Server, firewall, network, IT Hardware and Software faults

Everything that happens within your environment is configured to be stored as logs. These logs will fall into a category such as API Events, Monitoring Events, Log Events and Audit Events. These logs tell us things like what state each component is in currently, anything that happened recently, and will often include insights into how to fix a specific problem.

Why is this important? 

This event storage process allows us to investigate the information contained in these log files. It also provides the opportunity to automate remediation and facilitation processes for threats and opportunities to our environment. This article explains how to forward events from an API Connect solution into Splunk for efficient searching, monitoring and analysis of the data. Splunk can be thought of as a storage, processing and analyzing search engine for logs.

Prerequisites for this Task

  • API Manager 5.0.7.x or later installation. This can be an on prem deployment, IBM Bluemix Local, IBM Bluemix Reserved or IBM Bluemix Dedicated but not IBM Bluemix Publc.
  • A running configured Splunk instance.
  • A running Splunk HTTP Collector that is configured to connect to Splunk, accessible from the IBM API Connect Manager.
  • Splunk HTTP Authorization Token – HTTP Basic Auth token.
  • Request Channel – Generated GUID that Splunk uses to identify the channel.
  • If TLS is required to communicate with the HTTP Collector the certificates must be loaded into a TLS profile.

Instructions – Forwarding events from API Manager to Splunk

  1. Log on to API Cloud Manager with credentials to modify the settings.
  2. Navigate to Settings -> Analytics
  3. Under API Events, Click on ‘Export events to a third-party system’,
  4. Under Select Analytics Platform select ‘HTTP’ and click on – Configure’.
  5. Enter the following details into the form:
  • URL to the Splunk HTTP Collector
    • g. https://<Splunk HTTP Collector Host>:<port> /services/collector/raw
  • If TLS is required select Use TLS and select the TLS profile the certificate is stored in.
  • Add the additional headers as described in the table below.
Header Name Header Value
Authorization <Splunk HTTP Authorization Token>
x-splunk-request-channel <Request Channel>

Though this article explains how to do this via HTTP, it can also be done via syslog.

Please be aware that throughout the testing of this solution in API Connect, there were some problems encountered on API Manager. A short time after the ‘send event’ button is pressed, a notification popped up stating that the system failed to check if the event was sent to the third-party system successfully. If this occurs, please check Splunk for the test event. If configuration is correct in Splunk and API Manager, the data should be visible with the Splunk UI.

1 comment on"API Connect Integration with Splunk"

  1. Fabricio Lira August 20, 2017

    It would be great to see a similar article approaching usage of IBM Log Analysis or a more comprehensive solution integrating NOI & BSM. Just a suggestion to explore IBM portfolio

Join The Discussion

Your email address will not be published. Required fields are marked *