Text summarization and visualization using IBM Watson Studio

Summary

Automatic text summarization is part of natural language processing by which computers can understand, derive meaning and analyze human language. Text summaries can help reduce reading time, make the selection process easier, and improve the effectiveness of indexing. Text summarization algorithms are also less biased than human summarizers. Personalized summaries are useful in question-answering systems because they provide personalized information. Using automatic or semi-automatic summarization systems enable commercial abstract services to increase the number of texts they’re able to process.

Description

In this pattern, we’ll demonstrate a methodology to summarize and visualize text using IBM Watson Studio. Text summarization is the process of creating a short and coherent version of a longer document. There are two methods to summarize text: extractive and abstractive summarization. We’ll focus on extractive summarization which involves the selection of phrases and sentences from the source document to make up the new summary. Techniques involve ranking the relevance of phrases in order to choose only those most relevant to the meaning of the source. We’ll also demonstrate different methods to visualize the data that can provide a quick view.

After completing this code pattern, you’ll understand how to:

  • Summarize the text from documents and news feeds.
  • Create topic modeling on the text to extract important topics.
  • Create visualizations for better understanding of the data.
  • Interpret the summary and visualization of the data.
  • Analyze the text for further processing to generate recommendations or taking informed decisions.

Flow

flow

  1. Log into Watson Studio and create an instance, which includes object storage.
  2. Upload the data file to the object storage.
  3. Import a Jupyter Notebook from the URL.
  4. Run the processing techniques and create a statistical model for topics in the notebook.
  5. Explore the visualization in the notebook and export the output to object storage.

Instructions

Get the detailed instructions in the README file. These steps will show you how to:

  1. Create an account with IBM Cloud.
  2. Create a new Watson Studio project.
  3. Create the notebook.
  4. Add the data.
  5. Insert the credentials.
  6. Run the notebook.
  7. Analyze the results.
R K Sharath Kumar
Manjula Hosurmath