Overview

Skill Level: Intermediate

Watson Campaign Automation provides APIs for data controllers to fetch contact's data as part of Right of Access processing and submit erasure requests as part of Right to Erasure processing through a batched asynchronous job api. The responses from the asynchronous job are available for download for up to 3 days after which the GDPR job responses are deleted as they can have personal data.

Prerequisites

GDPR Lookup Keys must be specified within Watson Campaign Automation. 

It is recommended to collate the requests so that minimum calls are made to Watson Campaign Automation. Doing so will ensure that overall performance is not impacted for your organization.

Clients are responsible for ensuring their own compliance with various laws and regulations, including the European Union General Data Protection Regulation. Clients are solely responsible for obtaining advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulations that may affect the clients’ business and any actions the clients may need to take to comply with such laws and regulations. The products, services, and other capabilities described herein are not suitable for all client situations and may have restricted availability. IBM does not provide legal, accounting or auditing advice or represent or warrant that its services or products will ensure that clients are in compliance with any law or regulation.

Database GDPR Lookup Key Setup

Within your database, you are able to select up to 5 fields from the GDPR LookUp Fields screen that you will use for GDPR lookups. To configure this setup, take these steps:

  1. Click Settings on the Database details screen.

  2. Select up to 5 fields that you want to use as GDPR LookUp Fields by clicking the GDPR LookUp Fields tab.

Note: Lookup keys are a feature of Flexible Key databases and Email is used for the GDPR Lookup on Restricted databases.

Step-by-step

  1. Job Submission

    API Request

    Use the following API requests to call the GDPR Right of Access API.

    • POST /databases/{databaseId}/gdpr_access
      1. A Database Id must be provided in the path to identify the Database which the data subject will be located in. If the contact may exist in more than one database, a unique call must be made for each Database Id.

      2. The body of the POST must contain a CSV with one or more column names and the identifiers to be looked up for each column name. The content-type will be ‘text/csv’. See the example below:

        “MobileNumber”,“15063434670”

        “MobileNumber”,“15066754325”

        “Email”,”john@domain.com”

        “MobileNumber”,“18884266840”

        “Email”,”jill@domain.com”,

        “Email”,”jack@somedomain.com”

    GDPR Right Of Access Job Submission Against a Marketing Database

    Request Method

    POST

     

    Request URI

    /databases/{databaseId}/gdpr_access

     

    Sample URL

    https://api0.ibmmarketingcloud.com/rest/databases/10091/gdpr_access

     

    Request Headers

    Authorization: Bearer WCA_ACCESS_TOKEN

    Content-Type: text/csv

     

    Request Body – one identifier name-value pair on each line

    GDPR Lookup Key Column Name, GDPR Identifier Value

     

    Sample Request Body

    EMAIL,user1@eng.silverpop.com

    EMAIL,user2@eng.silverpop.com

    EMAIL,user99@eng.silverpop.com

    EMAIL,nonexistent@eng.silverpop.com

     

    Successful Response

    Status code: 202

    Response Header includes Location to poll for job status

    Sample value

    Location: https://api0.ibmmarketingcloud.com/rest/gdpr_jobs/32/status

     

    Response Body

    {

            “location”: “https://api0.ibmmarketingcloud.com/rest/gdpr_jobs/32/status”,

            “id”: 32

     }

  2. Status Check

    The Watson Campaign Automation /gdpr_access API will respond synchronously with a 202 – Accepted along with a response header named location which will have the URI for job status. (e.g. http://api5.ibmmarketingcloud.com/rest/gdpr_jobs/{jobId}/status)

    The GET method for /gdpr_jobs/{jobId} status returns a high level status that indicates whether processing is:

    • SUBMITTED
    • IN_PROGRESS
    • SUCCESS
    • FAILED

    When processing completes with a status of SUCCESS, the WCA status API response will also include a response header named ‘location’ having the URI for job results. (e.g. http://api5.ibmmarketingcloud.com/rest/gdpr_jobs/{jobId}/response)

    Note: An audit record of the GDPR request will be stored.

    For each identifier that was provided in the source file, a json body will indicate whether the contact was found and processed. If multiple contacts are found for the identifier, each result will be returned separately.

    The response will include the following:

    • An indication (true/false) of whether a contact was found
    • Columns organized as follows:
      • GDPR identifiers
      • Other known identifiers (Lookup Keys that are not marked as GDPR lookup keys)
      • Remaining database columns
    • Consent details for all relevant channels including
      • Channel (Email, SMS, Push)
      • Status (OPTED-IN, OPTED-OUT)
      • Consent Date
      • Qualifier (SMS Program Id or Mobile Push App Key)
    • Emails sent to the contact including:
      • Subject (not personalized)
      • Date/time of send
      • Whether the email was opened
      • Whether any links were clicked
    • Landing pages visited by the contact including:
      • Host
      • Number of visits
      • Date/time of first visit
      • Data/time of last visit
    • Universal Behaviors associated with a contact including:
      • Universal Behavior name
      • Number of behaviors
      • Date/time of first UB
      • Data/time of last UB

    Note: Universal Behaviors will include SMS and Mobile Push events that are stored as Universal Behaviors.

    Get Job Status

    Request Method

    GET

     

    Request URI

    /gdpr_jobs/{jobId}/status

     

    Sample URL

    https://api0.ibmmarketingcloud.com/rest/gdpr_jobs/32/status

     

    Request Headers

    Authorization: Bearer WCA_ACCESS_TOKEN

     

    Successful Response

    Status code: 303

    Response Header includes Location to fetch job result/response

    Sample value

    Location: https://api0.ibmmarketingcloud.com/rest/gdpr_jobs/32/response

     

    {

            “location”: “http://localhost:8081/rest/gdpr_jobs/32/response”,

            “status”: “SUCCESS”

    }

     

    Possible values for status: SUBMITTED, IN_PROGRESS, SUCCESS, FAILED

    Response Fetch

    Request Method

    GET

     

    Request URI

    /gdpr_jobs/{jobId}/response

     

    Sample URL

    https://api0.ibmmarketingcloud.com/rest/gdpr_jobs/32/response

     

    Request Headers

    Authorization: Bearer WCA_ACCESS_TOKEN

     

    Successful Response

    Status code: 200

  3. Response Fetch

    {

        “databaseId”: 10091,

        “timestamp”: “2018-03-21T20:04:14.963Z”,

        “contacts”: [

            {

                “contactFound”: true,

                “gdprIdentifiers”: [

                    {

                        “name”: “EMAIL”,

                        “value”: “user1@eng.silverpop.com”

                    }

                ],

                “data”: {

                    “otherIdentifiers”: [

                        {

                            “name”: “MobileNumber”,

                            “value”: “9500059000”

                        }

                    ],

                    “attributes”: [

                        {

                            “name”: “PackageName”,

                            “value”: “PLAN299”

                        },

                        {

                            “name”: “CustomerId”,

                            “value”: “1001”

                        }

                    ],

                    “consents”: [

                        {

                            “qualifier”: “”,

                            “channel”: “EMAIL”,

                            “consentDate”: “2018-02-06T19:48:35.000+00:00”,

                            “status”: “OPTED-IN”

                        },

                        {

                            “qualifier”: “500050”,

                            “channel”: “SMS”,

                            “consentDate”: “2018-02-19T17:29:59.000+00:00”,

                            “status”: “OPTED-IN”

                        }

                    ],

                    “emails”: [

                        {

                            “latestSendDate”: “2018-02-06T20:01:11.000+00:00”,

                            “subject”: “Welcome %%CustomerId%%”,

                            “earliestSendDate”: “2018-02-06T20:01:11.000+00:00”,

                            “fromName”: “Team Agni Admin”,

                            “fromAddress”: “teamagni@us.ibm.com”,

                            “opened”: false,

                            “clicked”: false

                        },

                        {

                            “latestSendDate”: “2018-02-06T20:45:48.000+00:00”,

                            “subject”: “NewsLetter %%MAILING_NAME%% %%MAILING_SEND_DAY%%”,

                            “earliestSendDate”: “2018-02-06T20:45:48.000+00:00”,

                            “fromName”: “Team Agni Admin”,

                            “fromAddress”: “teamagni@us.ibm.com”,

                            “opened”: false,

                            “clicked”: false

                        },

                        {

                            “latestSendDate”: “2018-03-12T08:09:25.000+00:00”,

                            “subject”: “%%MAILING_NAME%% – %%MAILING_SEND_DATE%%”,

                            “earliestSendDate”: “2018-03-12T08:09:25.000+00:00”,

                            “fromName”: “Team Agni Admin”,

                            “fromAddress”: “teamagni@us.ibm.com”,

                            “opened”: true,

                            “clicked”: true

                        }

                    ],

                    “webVisits”: [

                        {

                            “visitCount”: 2,

                            “host”: “localhost”,

                            “firstVisitDate”: “2018-03-12T08:23:52.914+00:00”,

                            “lastVisitDate”: “2018-03-12T08:28:26.635+00:00”

                        }

                    ]

                }

            },

            {

                “contactFound”: true,

                “gdprIdentifiers”: [

                    {

                        “name”: “EMAIL”,

                        “value”: “user2@eng.silverpop.com”

                    }

                ],

                “data”: {

                    “otherIdentifiers”: [],

                    “attributes”: [

                        {

                            “name”: “PackageName”,

                            “value”: “PLAN499”

                        },

                        {

                            “name”: “CustomerId”,

                            “value”: “1002”

                        }

                    ],

                    “consents”: [

                        {

                            “qualifier”: “”,

                            “channel”: “EMAIL”,

                            “consentDate”: “2018-02-06T19:48:46.000+00:00”,

                            “status”: “OPTED-IN”

                        }

                    ],

                    “emails”: [

                        {

                            “latestSendDate”: “2018-02-06T20:01:11.000+00:00”,

                            “subject”: “Welcome %%CustomerId%%”,

                            “earliestSendDate”: “2018-02-06T20:01:11.000+00:00”,

                            “fromName”: “Team Agni Admin”,

                            “fromAddress”: “teamagni@us.ibm.com”,

                            “opened”: false,

                            “clicked”: false

                        },

                        {

                            “latestSendDate”: “2018-02-06T20:45:48.000+00:00”,

                            “subject”: “NewsLetter %%MAILING_NAME%% %%MAILING_SEND_DAY%%”,

                            “earliestSendDate”: “2018-02-06T20:45:48.000+00:00”,

                            “fromName”: “Team Agni Admin”,

                            “fromAddress”: “teamagni@us.ibm.com”,

                            “opened”: false,

                            “clicked”: false

                        },

                        {

                            “latestSendDate”: “2018-03-12T08:09:25.000+00:00”,

                            “subject”: “%%MAILING_NAME%% – %%MAILING_SEND_DATE%%”,

                            “earliestSendDate”: “2018-03-12T08:09:25.000+00:00”,

                            “fromName”: “Team Agni Admin”,

                            “fromAddress”: “teamagni@us.ibm.com”,

                            “opened”: false,

                            “clicked”: false

                        }

                    ],

                    “webVisits”: [

                        {

                            “visitCount”: 2,

                            “host”: “localhost”,

                            “firstVisitDate”: “2018-03-12T08:23:52.914+00:00”,

                            “lastVisitDate”: “2018-03-12T08:28:26.635+00:00”

                        }

                    ]

                }

            },

            {

                “contactFound”: true,

                “gdprIdentifiers”: [

                    {

                        “name”: “EMAIL”,

                        “value”: “user99@eng.silverpop.com”

                    }

                ],

                “data”: {

                    “otherIdentifiers”: [],

                    “attributes”: [

                        {

                            “name”: “PackageName”,

                            “value”: “PLAN499”

                        },

                        {

                            “name”: “CustomerId”,

                            “value”: “1999”

                        }

                    ],

                    “consents”: [

                        {

                            “qualifier”: “”,

                            “channel”: “EMAIL”,

                            “consentDate”: “2018-02-06T19:48:56.000+00:00”,

                            “status”: “OPTED-IN”

                        }

                    ],

                    “emails”: [

                        {

                            “latestSendDate”: “2018-02-06T20:01:11.000+00:00”,

                            “subject”: “Welcome %%CustomerId%%”,

                            “earliestSendDate”: “2018-02-06T20:01:11.000+00:00”,

                            “fromName”: “Team Agni Admin”,

                            “fromAddress”: “teamagni@us.ibm.com”,

                            “opened”: false,

                            “clicked”: false

                        },

                        {

                            “latestSendDate”: “2018-02-06T20:45:48.000+00:00”,

                            “subject”: “NewsLetter %%MAILING_NAME%% %%MAILING_SEND_DAY%%”,

                            “earliestSendDate”: “2018-02-06T20:45:48.000+00:00”,

                            “fromName”: “Team Agni Admin”,

                            “fromAddress”: “teamagni@us.ibm.com”,

                            “opened”: false,

                            “clicked”: false

                        },

                        {

                            “latestSendDate”: “2018-03-12T08:09:25.000+00:00”,

                            “subject”: “%%MAILING_NAME%% – %%MAILING_SEND_DATE%%”,

                            “earliestSendDate”: “2018-03-12T08:09:25.000+00:00”,

                            “fromName”: “Team Agni Admin”,

                            “fromAddress”: “teamagni@us.ibm.com”,

                            “opened”: false,

                            “clicked”: false

                        }

                    ],

                    “universalBehaviors”: [

                        {

                            “lastBehaviorDate”: “2017-01-10T22:11:55.090+00:00”,

                            “count”: 1,

                            “firstBehaviorDate”: “2017-01-10T22:11:55.090+00:00”,

                            “Name”: “Twitter – Mentioned”

                        },

                        {

                            “lastBehaviorDate”: “2018-02-02T05:10:15.100+00:00”,

                            “count”: 4,

                            “firstBehaviorDate”: “2018-01-01T11:22:44.000+00:00”,

                            “Name”: “Twitter – Shared”

                        }

                    ],

                    “webVisits”: [

                        {

                            “visitCount”: 2,

                            “host”: “localhost”,

                            “firstVisitDate”: “2018-03-12T08:23:52.914+00:00”,

                            “lastVisitDate”: “2018-03-12T08:28:26.635+00:00”

                        }

                    ]

                }

            },

            {

                “contactFound”: true,

                “gdprIdentifiers”: [

                    {

                        “name”: “EMAIL”,

                        “value”: “user1@eng.silverpop.com”

                    }

                ],

                “data”: {

                    “otherIdentifiers”: [],

                    “attributes”: [

                        {

                            “name”: “PackageName”,

                            “value”: “PLAN99”

                        },

                        {

                            “name”: “CustomerId”,

                            “value”: “1100”

                        }

                    ],

                    “consents”: [

                        {

                            “qualifier”: “”,

                            “channel”: “EMAIL”,

                            “consentDate”: “2018-02-06T19:49:02.000+00:00”,

                            “status”: “OPTED-IN”

                        }

                    ],

                    “universalBehaviors”: [

                        {

                            “lastBehaviorDate”: “2017-11-30T03:40:55.010+00:00”,

                            “count”: 2,

                            “firstBehaviorDate”: “2017-11-04T09:00:30.010+00:00”,

                            “Name”: “Twitter – Favorited”

                        },

                        {

                            “lastBehaviorDate”: “2017-12-30T01:00:30.000+00:00”,

                            “count”: 2,

                            “firstBehaviorDate”: “2017-11-01T01:00:30.000+00:00”,

                            “Name”: “Twitter – Tweeted”

                        }

                    ],

                    “webVisits”: [

                        {

                            “visitCount”: 2,

                            “host”: “localhost”,

                            “firstVisitDate”: “2018-03-12T08:23:52.914+00:00”,

                            “lastVisitDate”: “2018-03-12T08:28:26.635+00:00”

                        }

                    ]

                }

            },

            {

                “contactFound”: false,

                “gdprIdentifiers”: [

                    {

                        “name”: “EMAIL”,

                        “value”: “nonexistent@eng.silverpop.com”

                    }

                ]

            }

        ]

    }

3 comments on"Watson – Performing GDPR Right of Access with Watson Campaign Automation APIs"

  1. Tom Coppedge May 30, 2018

    Base on my initial tests, it appears that this API only fetches data for 2018. Could someone verify that there is either a parameter to include data prior to 2018, or when that will be supported? It would be good to clearly provide any restrictions or limitations, along with future direction. Thanks.

    • Hello Tom,
      I am gathering this info for you and will get back to you as soon as I know more.

      Thank you
      Jeri

    • Hello Tom,
      What types of tests did you run? We have not put any restriction on the data that will be shared as part of ROA request.

      Thanks
      Jeri

Join The Discussion

Your email address will not be published. Required fields are marked *