In recent years, there has been a resurgence of privacy regulations worldwide. Enterprises are struggling to comply with complex regulatory requirements regarding data privacy as well as individuals' demand for the privacy of their data.

There are serious consequences of improper protection of data. Data breaches and non-compliance of regulatory requirements can cause fines and penalties to be imposed on the enterprises. In addition, they can result in a loss of customer loyalty, loss of revenue, and lawsuits, and can damage the enterprise's brand.

In this tutorial, you learn to protect enterprise data by using data privacy features such as data protection rules within the Watson Knowledge Catalog on IBM Cloud Pak for Data.

Learning objectives

In this tutorial, you:

Add data protection rules that enforce the restrictions that are specified in the policies and governance rules that you previously imported
Publish data to a governed catalog
Log in as various users to verify that the data protection rules are enforced

Prerequisites

IBM Cloud Pak for Data 4.6.
Watson Knowledge Catalog on Cloud Pak for Data.
Completion of the steps in the Analyze discovered data to gain insights on the quality of your data tutorial.

In addition, you must create two, non-admin users in your IBM Cloud Pak for Data instance to verify the results of applying the data privacy rules. In this tutorial, the user called regular_user represents regular users, and another called restricted_user represents users that have some additional access restrictions. Provide the User and Developer roles to both of the users so that they can access catalogs and assigned services.

Estimated time

It should take you approximately 60 minutes to complete the tutorial.

Step 1. Add data protection rule to deny access

Start by creating a data protection rule that prevents the users specified in the rule from viewing data.

Here, you create a data protection rule to enforce the "Restrict access for Passport and Driver's License number" governance rule, which says to restrict the visibility of assets with these fields to certain users (in this case, restricted_user).

Log in to your IBM Cloud Pak for Data instance.
Go to the Navigation Menu, expand Governance, and click Rules.
Click Add rule > New rule.
Select Data protection rule and then click Next.
Provide a name for the rule (Restrict access Passport and Driver's License), select the rule type of Access, and provide a description for the rule.
In the Rule builder section, create the first portion of Condition 1 by selecting and typing out the values to form the If condition of If Data class contains any Passport.

Click the + icon to add an additional section within the If condition. Then, choose Or from the drop-down list to join the two sections.

Create the second section of the If condition by typing and selecting the values to form Or Business term contains any Patient Driver's License."

By putting the two sections together, the full If condition now states that the rule should be run if the data class of an object is Passport or if the business term of the object is Patient Driver's License.
Click Add new condition + to add Condition 2 and choose And from the drop-down list to join the two conditions.
Create Condition 2 by selecting and typing the values to form the If condition of If User name contains any.
Click Add users +. In the dialog that appears, select restricted_user. Then, click Add.

Note: The restricted_user represents a user that exists in your system, whom you want to deny access to any data that has the Passport data class or the Patient Driver's License business term.

Condition 2 is now If User name contains any restricted_user, which means that the rule should be run if the user that is trying to access the data is the restricted_user.
Choose the Action to take as deny access to data.
Click Create. Your rule is saved and displayed in a new page.
Click Rules in the breadcrumb to return to the Rules page.

Step 2. Add data protection rule to redact data

Next, you experience how data can be redacted by using data protection rules. This method replaces the data in the column with a string of exactly 10 X characters. While this method helps to hide the data, it does not retain the original format of the data. Because all of the values are replaced with 10 X characters, it also loses referential integrity of the data, so if the column was used as a foreign key reference to some other table, that foreign key reference is lost when the data is redacted.

In this step, you create a data protection rule to enforce the "Redact Patient Birthdate" governance rule, which says that the patient birthdate should be kept hidden.

From the Rules page, click Add rule > New rule.
Select Data protection rule and then click Next.
Provide a name for the rule (Redact the Birthdate of Patient), select the rule type of Access, and provide a description for the rule.
In the Rule builder section, create Condition 1 by selecting and typing out the values to form the If condition of If Business term contains any Patient Birth Date.
Choose the Action to take as mask data. Click Create.
When new fields appear, update the Action by typing and selecting the values to form then mask data in columns containing Business term Patient Birth Date.

Within the Select how to mask data section, choose the Redact tile. By hovering over the value within the Redact tile, you see what the value looks like after masking.
Click Create. Your rule is saved and displayed in a new page.
Click Rules in the breadcrumb to return to the Rules page.

Step 3. Add data protection rule to substitute data

Now, you experience how data can be substituted by using data protection rules. This method replaces data with values that do not match the original format. However, if a value is used several times in a column with substituted data, it is replaced with the same substitution value. Thus, this method of masking data does not keep the data in the original format, but preserves referential integrity of the data.

In this step, you create a data protection rule to enforce the "Mask Sensitive personal information" governance rule, which says to mask sensitive personal information such as race, ethnicity, or gender.

From the Rules page, click Add rule > New rule.
Click Data protection rule.
Provide a name for the rule (Hide Sensitive Personal Information), select the rule type as Access, and provide a description for the rule.
In the Rule builder section, create Condition 1 by selecting and typing out the values to form the If condition of If Business term contains any Patient Race Patient Ethnicity Patient Gender.
Choose the Action to take as mask data. Click Create.
When new fields appear, update the Action by typing and selecting the values to form then mask data in columns containing Business term Patient Race Patient Ethnicity Patient Gender.

Within the Select how to mask data section, choose the Substitute tile. By hovering over the value within the Substitute tile, you see what the value looks like after masking.
Click Create. Your rule is saved and displayed in a new page.
Click Rules in the breadcrumb to return to the Rules page.

Step 4. Add data protection rule to obfuscate data

Next, you experience how data can be obfuscated by using data protection rules. This method replaces data with similarly formatted values. However, it does not preserve referential integrity or data distribution. This is a good method of masking financial account information, such as credit card or bank account numbers, government identity documents, such as passport or Social Security numbers, and personal contact details, such as phone numbers or email addresses.

In this step, you create a data protection rule to enforce the "Mask Social Security Number" governance rule, which says that the Social Security number should be replaced with dummy values.

From the Rules page, click Add rule > New rule.
Click Data protection rule.
Provide a name for the rule (Hide Social Security Number), select the rule type of Access, and provide a description for the rule.
In the Rule Builder section, create Condition 1 by selecting and typing out the values to form the If condition of If Data class contains any US Social Security Number.
Choose the Action to take as mask data.

Click Create.
When new fields appear, update the Action by typing and selecting the values to form then mask data in columns containing Data class US Social Security Number.

Within the Select how to mask data section, choose the Obfuscate tile. By hovering over and moving away from the value within the Obfuscate tile, you see that the data format for both the Before and After values are the same after masking.
Click Create. Your rule is saved and displayed in a new page.

Step 5. Publish assets to the default catalog

You completed the steps to discover and analyze your assets, and you also incorporated rules to protect your data. Now, you can publish your assets to a catalog to make these assets available to other users.

Go to the Navigation Menu, expand Governance, and click Data quality.

Click the HealthcareAnalysis project tile.
On the HealthcareAnalysis project page, select all of the asset checkboxes, and click Publish +.
In the Publish analysis results for datasets dialog, click Publish.
The assets publish to the default catalog. You can click the Refresh icon to refresh the table, and you should see that the last published dates for the assets are updated.
Go to the Navigation Menu, expand Catalogs, and click All catalogs.
On the Your catalogs page, click the Default Catalog tile.

The assets should now be available within the default catalog.

Default Catalog page

Step 6. Add collaborators to the default catalog

By default, only by the Admin user has access to the default catalog. You must add other users as collaborators to the catalog so they can access the assets within it.

Open the Access control tab.

Click Add collaborators +.
In the Add collaborators as Viewers dialog, choose the role that you want to provide to the new users (Viewer should suffice for this tutorial).

Within the Collaborators section, search for and select the users that you want to add as collaborators to the default catalog.

Click Add.

Note: You must add a minimum of two, non-admin users to the catalog. One of which you specified in the rule defined in Step 1.
The newly added users now appear in the list of collaborators on the Access control tab of the Default Catalog page.

Step 7. Verify the data protection rules are enforced

You can now log in as the non-admin users that you gave access to the default catalog and verify whether the data protection rules are enforced.

Log out of IBM Cloud Pak for Data and log back in as the restricted_user (the user specified in the rule you defined in Step 1).
Go to the Navigation Menu, expand Catalogs, and click All catalogs.
On the Your catalogs page, click the Default Catalog tile.
On the Default Catalog page, scroll to the catalog assets. Select the PATIENTS asset from the list.
An error message appears that says you cannot view the PATIENTS asset because it is blocked by the "Restrict access Passport and Driver's License" data protection rule.
Click Default Catalog in the breadcrumb to return to the Default Catalog page. If you try to access another asset, such as ENCOUNTERS, you should be able to see the contents of that asset. This is because the restricted_user is only denied access to the assets containing Driver's License or Passport fields, and none of them exist in the ENCOUNTERS asset.
Log out of IBM Cloud Pak for Data, and log back in as regular_user (the other user that you added to the default catalog as a collaborator).
As before, go to the Navigation Menu, expand Catalogs, and click All catalogs.
Click the Default Catalog tile.
On the Default Catalog page, scroll to the catalog assets. Select the PATIENTS asset from the list.

This time, the PATIENTS asset should load because the regular_user was not denied access to data.
On the PATIENTS data asset page, click the Asset tab to see the asset preview. It might take a while for the data to be masked, in which case, a notification appear.

You can see that five columns are masked. Click the Lock icon to view more details about the masked columns. Specifically, one column is obfuscated, three columns are substituted, and one column is redacted.
Look at the BIRTHDATE and SSN columns. Both of these have Lock icons next to the column names. This indicates that these columns are masked.

As before, you can click the Lock icons to view more details about the masked columns. For example, the Lock icon near BIRTHDATE says that the values and format in this column are redacted by the "Redact the Birthdate of Patient" data enforcement rule. All of the BIRTHDATE values are replaced by a string of 10 X characters. The SSN values are obfuscated. That is, they are replaced with other values of the same format. Thus, the real SSN values are hidden, but the formatting of the field is preserved.
Scroll horizontally to view the RACE, ETHNICITY, and GENDER columns.

The data in these columns are substituted with values that do not match the original format of the fields. However, you can see that within each of these columns, the same values occur multiple times. This is because all occurrences of a value in the column are replaced with the same text. This preserves the referential integrity of the column.

Summary

In this tutorial, you learned how to use data protection rules within Watson Knowledge Catalog on IBM Cloud Pak for Data. You added data protection rules to your healthcare data to limit the availability of the data to certain users. You added other data protection rules to hide some data by replacing it with other data. This ensured that users can view the columns that exist, while the real data is not visible to them. And based on the type of data masking used, they can also get an idea about the formats or table references of those columns.

Finally, you verified that the data protection rules are enforced. You published the data assets to the default catalog and verified that you have access to the assets and the data by logging in as other users who are not owners of the data asset.

This tutorial is part of the An introduction to the DataOps discipline series.

Protect your data using data privacy features

Protect your data using data privacy features