Overview

Skill Level: Any Skill Level

This recipe explains a method to retrieve performance data from monitored devices using ScienceLogic out of the box APIs.

Ingredients

For this recipe the prerequisite are the following:

ScienceLogic API connectivity (default port 443);
Administrator access on ScienceLogic to create an new user (for API queries);
An external tool for making the external queries (e.g. Postman or cURL).

Step-by-step

  1. Create a new user on the ScienceLogic tool

    In order to run queries against the ScienceLogic API a new user may be created for better user management and to set the right permissions.

    To create a new user on ScienceLogic follow steps below:

    1) Log in to ScienceLogic portal using an admin ID (e.g. em7admin)

    2) Go to Registry > Accounts > User Accounts

    3) Create on the “Create” button

    4) Fill the basic information including “First Name” and the “Password” as shown below

      img1

    5) Click “Save” button – a new page will be displayed as shown below

     img2

    6) Set the Organization the API user needs to access (Primary + Additional Organization Menberships)

    7) Set “Basic User Privileges”, “Events – View”, “Devices – Information View”, “Interfaces – View”, “Monitors – View” as the basic privileges for the user to access API data for events and devices.

    8) Click Save

  2. Configuring an external tool to query SL1 API

    On this example I am using the Postman tool to query SL1 API.

    To configure the authentication on Postman navigate to Authentication and set the user/password as shown below.

     img3

    For the URL, insert the SL1 URL and include “/api” to list available API queries and test the authentication.

    Hit “Send” button and the status 200 OK show appears.

  3. API calls - General Information

    Here are some generic information for using API queries.

    1) The API calls are pointed to SL1 URL + /api

    When the generic “/api” URL is used, result is a list of available child URLs that can be used to get more detailed information. This same concept applies to all other API URLs.

    Example:

    SL1 URL = https://api.gtslab-sciencelogic.ams

    API URL = https://api.gtslab-sciencelogic.ams/api

     

    2) Getting details of some specific device:

    When the “/api/device” URL is used a list of all devices is presented. To get information of some specific device you need to specify the device ID.

    Example:

    List of devices = https://api.gtslab-sciencelogic.ams/api/device

    Details of a specific device ID 38 = https://api.gtslab-sciencelogic.ams/api/device/38

    Note: The IDs used on the queries are based on the SL1 environment and may be different on different SL1 installations.

     

    3) Query options:

    Every API query has a specific set of options that can be listed just quering the API. The options will be displayed as part of the query output.

    Example below:

     img4

    On this query, we can use – as example – hide_filterinfo=1 and extended_fetch=1 to hide information about filters and improve device information data.

    The URL would be like following:

    For all the devices: https://api.gtslab-sciencelogic.ams/api/device?hide_filterinfo=1&extended_fetch=1

    For device ID 38: https://api.gtslab-sciencelogic.ams/api/device/38?hide_filterinfo=1&extended_fetch=1

    Result can include other URLs, for Credentials, Organization and others.

    Information for those URLs can be obtained pointing query to each individual URL.

    For example:

    Query output for device ID 38 includes “snmp_cred_id” = “/api/credential/snmp/57”. Querying the credential URL “/api/credential/snmp/57” will give details on that credential.

  4. API query to get performance metrics from traditional devices (e.g Linux OS)

    As shown above, API query to URL “https://api.gtslab-sciencelogic.ams/api/device/38?hide_filterinfo=1&extended_fetch=1” will retrieve data from device ID 38.

    Result will eventually includes other URLs, as shown below.

    “applications”: {
    “URI”: “/api/device/38/aligned_app”,
    “description”: “Aligned Dynamic Applications”
    },
    “performance_data”: {
    “URI”: “/api/device/38/performance_data”,
    “description”: “Collected Performance Dynamic App Data”
    },
    “config_data”: {
    “URI”: “/api/device/38/config_data”,
    “description”: “Collected Config Dynamic App Data”

     

    To get performance data from device ID 38 use URL https://api.gtslab-sciencelogic.ams/api/device/38/performance_data?hide_filterinfo=1&extended_fetch=1

    It will also includes link to other URLs for more specific query.

    For example:

    “appname”: “Host Resource: Storage”,
    “presentations”: [
    {
    “presname”: “% Storage Used”,
    “data_found”: “1”,
    “collection_enabled”: “1”,
    “data”: {
    “URI”: “/api/device/38/performance_data/185/data?duration=24h”,
    “description”: “Host Resource: Storage:% Storage Used”
    },
    “normalized_hourly”: {
    “URI”: “/api/device/38/performance_data/185/normalized_hourly?duration=7d”,
    “description”: “Host Resource: Storage:% Storage Used (Hourly Normalized)”
    },
    “normalized_daily”: {
    “URI”: “/api/device/38/performance_data/185/normalized_daily?duration=30d”,
    “description”: “Host Resource: Storage:% Storage Used (Daily Normalized)”
    }
    },

     

    Above result includes the appname which is the Dynamic Application (DynApp) that is collecting data for that device, in this example “Host Resource: Storage”.

    Note: ID 185 is the Presentation ID for the “Host Resource: Storage:% Storage Used” metric.

    To get last 24h of “% Storage Used” from device ID 38 collected by the DynApp “Host Resource: Storage”, use following query.

    URL: https://api.gtslab-sciencelogic.ams/api/device/38/performance_data/185/data?duration=24h&hide_filterinfo=1&hide_options=1&extended_fetch=1

    Note: Output will contains a list of values by time interval in epoch format for the last 24h.

     

    Another example, now getting CPU information for device ID 38.

    To get last 2h of “Physical Memory Utilization” from device ID 38 use following API call.

    Note: ID 6648 is the Presentation ID for the “Net-SNMP: Physical Memory:Physical Memory Utilization” metric.

    URL: https://api.gtslab-sciencelogic.ams/api/device/38/performance_data/6648/data?duration=2h&hide_filterinfo=1&hide_options=1&extended_fetch=1

    Result will be like below

    img5

  5. API query to get performance metrics from cloud devices (e.g AWS EC2)

    Same as before for Linux OS, API queries for AWS or any other component works similar. The only difference will be the IDs for the query.

    Using API URL “https://api.gtslab-sciencelogic.ams/api/device/?hide_filterinfo=1” we can see all devices and identify devices we want to get information from. As the result is large we may need to filter by some field, maybe Device Class.

    To target the right AWS devices we may filter devices by Device Class, as shown below (e.g. EC2):

    https://api.gtslab-sciencelogic.ams/api/device_class?limit=100&extended_fetch=0&hide_filterinfo=1&filter.0.description.contains=EC2

    Based on the provided list, we can identify a class named “EC2 Instance micro” which has ID 660.

    To filter devices by that class use: https://api.gtslab-sciencelogic.ams/api/device?hide_filterinfo=1&limit=100&extended_fetch=0&filter.0.class_type%2Fdescription.eq=EC2%20Instance%20micro

    The result will be as shown below:

     img6

     

    Now for each device we can get more information just like explained before.

    As we can see in the result of query above, device ID 1265 is one possible target device of Class EC2 Instance Micro.

    We can get its performance data details using following API URL:

    URL: https://api.gtslab-sciencelogic.ams/api/device/1265/performance_data?hide_filterinfo=1&limit=100&extended_fetch=1

    As part if the result of the query above we can see following information:

    “appname”: “AWS EC2 Instance Performance”,
    “presentations”: [
    {
    “presname”: “CPU Utilization”,
    “data_found”: “1”,
    “collection_enabled”: “1”,
    “data”: {
    “URI”: “/api/device/1265/performance_data/7607/data?duration=24h”,
    “description”: “AWS EC2 Instance Performance:CPU Utilization”
    },
    “normalized_hourly”: {
    “URI”: “/api/device/1265/performance_data/7607/normalized_hourly?duration=7d”,
    “description”: “AWS EC2 Instance Performance:CPU Utilization (Hourly Normalized)”
    },
    “normalized_daily”: {
    “URI”: “/api/device/1265/performance_data/7607/normalized_daily?duration=30d”,
    “description”: “AWS EC2 Instance Performance:CPU Utilization (Daily Normalized)”
    }
    },

    Note: ID 7607 is the Presentation ID for the “AWS EC2 Instance Performance:CPU Utilization” metric.

    So we can now use this information for a new query addressed to a specific data (e.g. CPU Utilization)

    The URL would be: https://api.gtslab-sciencelogic.ams/api/device/1265/performance_data/7607/data?duration=2h&hide_filterinfo=1&limit=100&extended_fetch=1&hide_options=1

    Result is a time series data as shown below.

     img7

  6. Getting information for multiple devices in one query

    Warning: This method should be used very carefully as the query may impact performance on SL1 and Network.

    To target on a specific Presentation ID instead of a device (see examples of Presentation IDs on previous sections) you can use an API URL like below.

    URL: https://api.gtslab-sciencelogic.ams/api/data_performance_raw/device/dynamic_app?duration=1h&presentation_objects=7607&hide_filterinfo=1

    The output will be a list of server and time series data as shown below:

     img8

    Note: in this example the device will be presented as an ID. To get the name and other details of that device you need to use another query like below (e.g. device ID 11366)

    URL: https://api.gtslab-sciencelogic.ams/api/device/11366

     

  7. Simple Python code

    Here is a simple code in Python to extract information from SL1 API.

     

    import requests, json

    #Define SL1 Base API URL
    baseURL="api.gtslab-sciencelogic" ### ***Change to your own URL - this is just an example***

    def CALL_API(URL):
    payload = {}
    headers = {'Authorization': 'Basic YX23NlcjpkxxMHZu90=='} ### ***Change to your own Key***
    response = requests.request("GET", URL, headers=headers, verify=False, data = payload)
    parsed_json = (json.loads(response.text))
    #print(json.dumps(parsed_json, indent=1, sort_keys=False))
    #print(parsed_json[0].keys())
    return parsed_json

    #Define API Call parameters
    duration="30m"
    limit="-1"
    Metric_ID="5395" ### ***Change to the Metric ID you want on your own SL1 env***
    Metric_Label="Linux: Memory Stats:Physical Memory Utilization" ### ***Put your metric label in here***
    apiPath="/api/data_performance_raw/device/dynamic_app"
    url = "https://"+baseURL+apiPath+"?hide_filterinfo=1&duration="+duration+"&limit="+limit+"&presentation_objects="+Metric_ID
    result = CALL_API(url)

    print()
    #print("API: "+url)
    print("Metric ID: "+Metric_ID+" = "+Metric_Label)
    print("Period: "+duration)
    print()
    print("Device_ID,Device_Name,Metric_ID,Collection_Time,Raw_Data")
    for n in result:
    #print(n)
    apiPath = str(n['device'])
    url = "https://"+baseURL+apiPath
    device = CALL_API(url)
    Device = device["name"] Presentation = str(n['presentation'])
    for d in n['data']:
    Epoch_Time = str(d[0])
    Value = str(d[1])
    print(apiPath + "," + Device + "," + Presentation + "," + Epoch_Time + "," + Value)

     

    Script output (on my SL1 system) **values were changed to preserve security.

    Metric ID: 5395 = Linux: Memory Stats:Physical Memory Utilization
    Period: 30m

    Device_ID,Device_Name,Metric_ID,Collection_Time,Raw_Data
    /api/device/19066,2.17.12.2,5395,1580906700,79.637
    /api/device/19066,2.17.12.2,5395,1580907000,83.113
    /api/device/19066,2.17.12.2,5395,1580907300,86.893
    /api/device/19066,2.17.12.2,5395,1580907600,90.049
    /api/device/19066,2.17.12.2,5395,1580907900,93.072
    /api/device/19143,2.17.12.3,5395,1580906700,66.869
    /api/device/19143,2.17.12.3,5395,1580907000,66.952
    /api/device/19143,2.17.12.3,5395,1580907300,66.842
    /api/device/19143,2.17.12.3,5395,1580907600,67.202
    /api/device/19143,2.17.12.3,5395,1580907900,67.142
    /api/device/19144,2.17.12.4,5395,1580906700,98.631
    /api/device/19144,2.17.12.4,5395,1580907000,98.972
    /api/device/19144,2.17.12.4,5395,1580907300,98.697
    /api/device/19144,2.17.12.4,5395,1580907600,98.67
    /api/device/19144,2.17.12.4,5395,1580907900,98.614
    /api/device/23596,29.9.16.4,5395,1580906700,0
    /api/device/23596,29.9.16.4,5395,1580907000,0
    /api/device/23596,29.9.16.4,5395,1580907300,0
    /api/device/23596,29.9.16.4,5395,1580907600,0
    /api/device/23596,29.9.16.4,5395,1580907900,0
    /api/device/24080,production,5395,1580906700,91.449
    /api/device/24080,production,5395,1580907000,91.458
    /api/device/24080,production,5395,1580907300,91.465
    /api/device/24080,production,5395,1580907600,91.464
    /api/device/24080,production,5395,1580907900,91.459

Join The Discussion