Get started with the Semantic Data Science API

The Semantic Data Science API is part of the early access offering from IBM Research. It packages some of the state-of-the-art techniques for doing column-to-concept mapping, code analysis, and knowledge extraction along with a unified framework to access the techniques.

Get started with the Semantic Data Science API

The Semantic Data Science API provides access to the column-to-concept mapping, code analysis, and knowledge extraction methods in an easy-to-use manner. The API service is hosted in the IBM Developer API Hub platform, where you can get a free trial to easily try the service.

This tutorial provides a step-by-step guide on how to use the API service to augment data to build AI solutions with domain knowledge that typically requires the painstaking efforts of skilled data scientists. The tutorial uses Python code snippets, but you can choose your own programming language to invoke the APIs.

Currently, the service supports the following endpoints:

• Data Management
  • GET /api/datasets
  • GET /api/columns/{dataset}
  • GET /api/column_types/{dataset}
  • GET /api/dataset/{dataset}/size
  • GET /api/dataset/{dataset}/{from}/{to}
• Concept Mapping
  • GET /api/concepts/{datasets}/candidates/{columnId}
  • POST /api/concepts/{dataset}
  • GET /api/concepts/{dataset}
• Code Mining
  • GET /api/domain/concepts/{dataset}
  • GET /api/domain/knowledge/{dataset}
  • GET /api/formulas/{dataset}
  • POST /api/formulas/{dataset}

You can take a look at the comprehensive API documentation.

Prerequisites

To complete this tutorial, you need:

• An IBMid
• An IBM Cloud account
• Python 3 and pip

Estimated time

It should take you approximately 20 minutes to complete this tutorial.

Steps

Step 1. Prepare a Cloud Object Storage service instance (Free Trial)

1. Log in to your IBM Cloud account. If you do not have an IBM Cloud account, you can sign up for one.

2. Open the catalog, and search for Cloud Object Storage.

3. Create a Cloud Object Storage instance, and choose the Free Lite plan. Change the name of the instance if you want, then click Create.

4. To open your Cloud Object Storage instance, select Resource List in the main menu.

5. Click Storage, and select your instance name.

6. In the Cloud Object Storage menu, click Buckets, then click Create bucket.

   

7. You can create a custom bucket, or choose from the IBM predefined configurations. For this tutorial, select Customize your bucket.

   

   1. To create a bucket, start by choosing a unique name.

   2. Choose the level of resiliency that you want. Then, choose a location where you would like your data to be physically stored.

      Resiliency refers to the scope and scale of the geographic area across which your data is distributed. Cross Region resiliency spreads your data across several metropolitan areas, while Regional resiliency spreads data across a single metropolitan area. A Single Data Center distributes data across devices within a single site only.

   3. Choose the bucket's storage class to accurately reflect how often you expect to read the stored data

      

   4. Click Create bucket.

      

      After you click Create bucket, it gives you a success message and redirects to the page where you can upload the sample data, or you can follow step 8 to upload data.

8. Open your bucket and upload the sample data file.

   

9. Open the bucket configuration. In the Endpoints tab, copy the public endpoint URL and save the URL to use later in this tutorial.

10. To create a new API key, go to IBM Cloud Manage > Access (IAM) > API keys >. Click Create one now.

    

11. You are redirected to a new page. Click Create an IBM Cloud API key.

    

12. Enter a name and description for your API key, and click Create.

    

13. Click Show to display the API key, or click Copy to copy the API key to use later in this tutorial.

    

14. Assign READ/WRITE access to the new Cloud Object Storage service instance to the newly created API key.

Step 2. Authentication

Each API request requires authentication information in the request header. The authentication credential includes the Client ID and the Client secret.

To create a new credential:

1. Click Get trial subscription on the Semantic Data Science API Page.

2. Subscribe to a free trial by using your IBMid. If you don’t have a IBMid, create one during the process.

3. Log in with your IBMid, and navigate to the dashboard of your developer profile.

4. Click APIs, then click Semantic Data Science.

5. In the Key Management, expand the drop-down list of API keys. The Client ID and Client Secret are ready to use.

Step 3. Invoke Semantic Data Science APIs

This section walks you through how to invoke APIs to submit a call to the Semantic Data Science APIs and retrieve concepts of a domain. The following code snippets are written in Python.

1. Install the required Python package by using pip.

    > pip install requests
   

2. Use the following code snippet to invoke the API to retrieve the concepts from the domain of your data set. Replace the content of headers with the credentials that you obtain from Step 1 and Step 2.

   Note: All of the parameters available for domain concept retrieval are discussed in the API Hub documentation.

    import http.client
   
    conn = http.client.HTTPSConnection("dev.api.ibm.com")
   
    headers = {
      'X-IBM-Client-Id': "REPLACE_THIS_VALUE",
      'X-IBM-Client-Secret': "REPLACE_THIS_VALUE",
      'X-IBM-COS-Api-Key': "REPLACE_THIS_VALUE",
      'X-IBM-COS-Endpoint-Url': "REPLACE_THIS_VALUE",
      'X-IBM-COS-Bucket': "REPLACE_THIS_VALUE",
      'accept': "application/json"
    }
   
    conn.request("GET", "/automatedai/test/api/domain/concepts/REPLACE_DATASET", headers=headers)
   
    res = conn.getresponse()
    data = res.read()
   
    print(data.decode("utf-8"))
   

When the request completes, the domain concepts with their relevance score are returned in the response. The following code snippet is an example of a response.

```
{
  "business": 15,
  "car": 47,
  "fuel": 34,
  "fuel economy": 17,
  "fuel type": 29,
  "gas": 14,
  "mile": 19,
  "prices": 15,
  "rate": 25,
  "vehicle": 44
}
```

Step 4. Other operations in the API suite

Other than retrieving the domain concepts described in the previous step, the API service supports column-to-concept mapping and code mining. You can read about those operations in the API documentation. In the IBM Developer API Hub, you can also invoke the APIs through the UI by clicking Try this API under each code snippet. There is a full getting-started tutorial that walks you through the steps.

Summary

This tutorial discussed how to subscribe to a free trial for the Semantic Data Science API and how to invoke the API hosted in IBM Cloud. If you have any questions, check the API documentation or contact us.