Current natural language processing techniques cannot extract or interpret data as required by a domain or industry because the data (entities) represents different meanings in different domains. With this code pattern, you’ll learn how to develop a solution that can help using Watson™ Knowledge Studio (WKS) and Watson Natural Language Understanding (NLU).
This code pattern describes how to analyze SMS messages using Watson Knowledge Studio and Watson Natural Language Understanding to extract entities in the data. Specifically, the code pattern explains how to use Watson Knowledge Studio to create and train a machine learning model using human annotated documents, integrating the machine model into an NLU service, and extracting domain-specific entities using this NLU service.
The SMS messages in this code pattern are related to merchants offering special offers to their customers. With NLU, you can extract some general information from each text, but you might want to add the capability to extract additional specific data, such as what the offer is, who the merchant is, how long the offer is valid, and what the merchant’s phone number and website is. You can accomplish this by loading sample messages into WKS and training it to recognize entities within each text. The result is a model that you can then use to process additional messages.
After completing this code pattern, you should know how to:
- Upload a corpus with WKS
- Import types to WKS
- Use WKS to create a model
- Deploy a WKS model to NLU
- Call NLU APIs with a WKS model specified
- Load type system and corpus files into Watson Knowledge Studio.
- Generate a model by training and evaluating data.
- Deploy the WKS model to Watson NLU.
- Provides an SMS message to the app for analysis.
- Watson NLU analyzes the SMS message for processing and returns extracted domain-specific entities based on the WKS model.
Find the detailed steps for this pattern in the README. The steps will show you how to:
- Clone the sms-analysis-with-wks repo.
- Create IBM Cloud services.
- Create a Watson Knowledge Studio workspace.
- Upload the type system.
- Import corpus documents.
- Create an annotation set.
- Create a task for human annotation.
- Create the model.
- Deploy the machine learning model to NLU.
- Test the model with cURL.
- Run the application.