PII extraction using pretrained models

Personal identifiable information (PII) extraction refers to the process of identifying and extracting personal information from various sources, such as documents, databases, and websites. PII is any information that can be used to identify an individual, including their name, address, phone number, email address, social security number, driver's license number, and credit card number. PII extraction is a crucial process in maintaining data privacy compliance while also extracting valuable insights from data.

IBM Watson NLP is a standard embeddable AI library that is designed to tie together the pieces of IBM Natural Language Processing. It provides a standard base natural language processing (NLP) layer along with a single integrated roadmap, a common architecture, and a common code stack designed for widespread adoption across IBM products. The watson_nlp library is available on IBM Watson Studio as a runtime library so that you can directly use it for model training, evaluation, and prediction.

PII extraction is a crucial process in maintaining data privacy compliance while also extracting valuable insights from data. IBM Watson NLP models offer a powerful solution for PII extraction, utilizing natural language processing and machine learning techniques to accurately identify and extract personally identifiable information. Additionally, these models offer a flexible and scalable solution, allowing businesses to fine-tune the models to extract specific types of PII and integrate them into their applications for compliance with data privacy regulations. These models can help businesses improve operational efficiency by automating manual processes, reducing errors, and saving time and resources.

The following figure shows the Watson NLP architecture.

Learning objectives

This tutorial provides an introduction to IBM Watson NLP, and guides you through the process of using pretrained models for PII extraction.

Prerequisites

To follow the steps in this tutorial, you need:

• An IBMid
• A Watson Studio project
• A Python pretrained models notebook
• Your environment set up

Estimated time

It should take you approximately 60 minutes to complete this tutorial.

Steps

The tutorial demonstrates the extraction of PII using pretrained Watson NLP models. This section focuses on PII extraction models for the following PII entities using pretrained models. Refer to this notebook for extracting custom PIIs using fine-tuned models.

Step 1. Generate the testing data

1. Generate the sample data set for a name, a credit card number, and a social security number by using a Faker library.

   fake = Faker(locale='en_US')
   
   def format_data():  
           #Generate a random
           name = fake.name() 
   
           #Generate a random SSN 
           ssn = fake.ssn()
   
           #Generate a random CCN 
           ccn = fake.credit_card_number()
   
           text_1 = """My name is %s, and my social security number is %s. Here's the number to my Visa credit card is %s""" % (name, ssn, ccn)
   
           text_2 = """%s is my social security number. The name on my credit card %s is %s."""% (ssn, ccn, name)
   
           text_3 = """My credit card number is %s and social security number is %s, I am %s""" %(ccn,ssn,name)
   
           text = random.choice([text_1, text_2,text_3])
   
           return text
   
   text = format_data()
   print(text)
   
   'My credit card number is 4084897820169892970 and social security number is 178-32-7893, I am Theresa Bowen'
   

Step 2. PII extraction

The process of PII extraction involves identifying and extracting specific pieces of information, such as names, social security numbers, credit card details, and locations from input text. This is achieved through by using an entity-mentions block, which encapsulates various algorithms that are designed to perform this task.

The entity-mentions block provides implementations of strong PII extraction algorithms from four different families: rule-based, classic machine learning, deep-learning, and transformers. These algorithms use different techniques and approaches to identify and extract PII from text, allowing for a more robust and comprehensive PII extraction process.

By using the entity-mentions block and its various algorithms, you can ensure that you are effectively and accurately extracting PII from your data, which improves data privacy and security.

The process of identifying PII entities from text can be accomplished by using two different types of models:

1. A rule-based model, also known as the RBR model, which is designed to handle syntactically regular entity types such as numbers, emails, and phone numbers. This model relies on predefined rules to identify and extract these entities from text.

2. A model that is trained on labeled data for the more complex entity types such as persons, organizations, and locations. This model uses machine learning techniques to learn patterns and relationships between words and their corresponding entity types to accurately identify and extract entities from text.

2.1 PII extraction function

Rule-based models can be directly applied to input text without any dependency on preprocessing blocks. However, models that are trained from labeled data, such as BiLSTM and SIRE, require the syntax block to be run first to generate the expected input for the entity-mentions block.

Therefore, to perform PII extraction using these models, you must load the syntax model and three PII extraction models.

# Load a syntax model to split the text into sentences and tokens
syntax_model = watson_nlp.load(watson_nlp.download('syntax_izumo_en_stock'))
# Load bilstm model in WatsonNLP
bilstm_model = watson_nlp.load(watson_nlp.download('entity-mentions_bilstm_en_pii'))
# Load rbr model in WatsonNLP
rbr_model = watson_nlp.load(watson_nlp.download('entity-mentions_rbr_multi_pii'))
# sir Load rbr model in WatsonNLP
sire = watson_nlp.load(watson_nlp.download('entity-mentions_sire_en_stock-wf'))

2.2 Run the pretrained models for PII extraction

• BiLSTM pretrained: The term "pretrained" refers to a pretrained BiLSTM model, which has already been trained on a large corpus of text data and can be fine-tuned or used as-is for specific NLP tasks, such as PIIs or entities extraction. By using a pretrained BiLSTM model, you can use the knowledge learned from the training data to quickly build NLP applications with improved accuracy.

  The following image shows how the pretrained BiLSTM model can detect the person name as Lori Gross.

  

• RBR pretrained: A pretrained rule-based model is a model that has already been trained on a large corpus of text data and has a set of predefined rules for processing text data. By using a pretrained rule-based model, you can use the knowledge learned from the training data to quickly build NLP applications with improved accuracy.

  The following image shows how a pretrained RBR Model can detect credit card numbers and social security numbers.

  

Conclusion

This tutorial explained how to use the Watson NLP library and how easily you can run various PII extraction using pretrained models on input text. As a next step, work through the notebook to try out this feature.