Infuse generative AI into document processing

IBM Datacap is a robust data capture solution that is designed to streamline the transformation of unstructured information into actionable data. It uses advanced optical character recognition (OCR) technology, machine learning, and natural language processing (NLP) to accurately extract and classify data from a variety of sources, such as scanned documents, emails, and PDFs.

However, IBM Datacap faces challenges in handling diverse document formats, maintaining high accuracy in data extraction, and managing complex document layouts. Continuous training and updates are required to keep pace with evolving data types, which can be resource intensive. So, IBM Datacap is turning to the generative AI capabilities in watsonx.ai for help. IBM watsonx.ai is part of the watsonx platform, which brings together generative AI capabilities powered by large language models (LLMs) and traditional machine learning.

Generative AI can significantly enhance data classification and extraction by providing more accurate and adaptive optical character recognition (OCR) capabilities. It can automatically recognize and adapt to new document formats and layouts, reducing the need for constant manual updates. By improving data classification and validation processes, generative AI ensures higher accuracy and efficiency in data capture, adding substantial value to IBM Datacap capabilities.

By integrating Datacap with LLMs that now perform document classification and data extraction, developers experience faster application development, deeper insights from complex documents, and more reliable output because the automated process reduces the scope of human errors. Users of these Datacap applications experience improved accuracy, templateless capture (the solution is vendor agnostic), and a more efficient processing of large volumes of documents (applications scale much more easily).

In this tutorial, you’ll learn how to integrate IBM Datacap with large language models (LLMs). It explains how to configure watsonx.ai actions into your existing Datacap configuration. You’ll also learn how to set up a watsonx.ai service on IBM Cloud, specify the service information in a Datacap application to call the watsonx.ai service so that you can take advantage of the generative AI capabilities of watsonx.ai.

Prerequisites

You need an IBM Cloud account to create a watsonx.ai project.

Also, you need IBM Datacap, version 9.1.9, upgraded to IFIX005. You can download IFIX005 from IBM Support to upgrade your current installation. Run the Update.exe file first, and then run the UpdateOCR.exe file.

This tutorial uses the Lite plan for the IBM Cloud services, which the initial amount of tokens and computation hours are limited. At the time of writing this tutorial, the Lite tier has 50,000 tokens per month that can be used for chatting and prompt engineering with the various large language models, and 20 capacity unit hours (CUH) for machine learning use. One word in the input or output to or from a model consumes approximately three to five tokens. Additionally, Tuning Studio is not available in the Lite tier at all. If you need more tokens or compute hours, you can upgrade your account, but for the purposes of completing this tutorial, the Lite tier should suffice.

Steps

Before you dive in to the steps in this tutorial, watch me demonstrate how to integrate Datacap with LLMs using watsonx.ai to improve accuracy and be more efficient with processing large volumes of documents:

Step 1. Create a watsonx.ai project

1. Sign up for the watsonx.ai free trial. Then, log in to watsonx.ai using your IBM Cloud account.

2. Create a watsonx.ai project.

   Take note of the project ID in Project > Manage > General > Project ID. You'll need this ID to be able to call the watsonx.ai service from Datacap.

3. Create a Watson Machine Learning (WML) service instance (choose the Lite plan, which is a free instance).

4. Associate the WML service to the project you created in watsonx.ai.

5. Generate an API Key. Save this API key, as you will need it to be able to call the watsonx.ai service from Datacap.

6. Open the watsonx.ai Prompt Lab. You need to copy these IDs to be able to call the watsonx.ai service from Datacap:

   • The Prompt Lab curl command will be used as the EndpointURL value.
   • The ID for the large language model will be used as the Model value.

   • The ID for your watsonx.ai project is also in this Prompt Lab code, if you did not save it earlier.

     You will specify these IDs or keys at the end of the next step.

Step 2. Configure watsonx.ai actions in Datacap

1. Assuming you already have IFIX005 installed as part of prerequisites, run the setup here to install watsonx_ai action library.

2. To verify the installation of the new action library, log in to the IBM Datacap Studio using the credentials for taskmaster login for the APT application.

   

3. In the Actions library, scroll down to see the watsonx_ai actions. Expand the watsonx_ai actions group to see the actions.

   

   If you do not see watsonx_ai in the actions library, ensure that you have the proper version of IBM Datacap installed (see the Prerequisites section above).

4. To be able to call the watsonx.ai service from Datacap, you must specify values for these parameters for the watsonx_ai actions:

   • SetAccessTokenURL. This URL is used to create an access token. If not specified, it uses the default URL, iam.cloud.ibm.com/identity/token. We will use the default URL, so no need to call the action.
   • SetAPIKey. This is the API key for the watsonx.ai project that you created in the previous step.
   • SetEndpointURL. This is the Prompt Lab curl command you copied in the previous step.
   • SetModel. This is the value of the model_id parameter that you copied from the Prompt Lab curl code in the previous step.

   • SetProjectID. This is the ID for the watsonx.ai project that you made note of when creating the project in the previous step.

     Edit each parameter and specify the values in parentheses after it.

     

Step 3. Configure the watsonx.ai sample application in Datacap Studio

1. Download the watsonx.ai sample application

   If you have installed the latest Datacap release with IFIX005, this application should be already present in C:\Datacap folder.

2. Extract the files, and copy the application named watsonxai in to the C:\Datacap folder.

   

3. In the C:\Datacap folder, edit the datacap.xml file and add the following XML:

   <?xml version="1.0" encoding="UTF-8" standalone="yes">
    <datacap ver="9.0">
      <app name="TradeFinance" ref="C:\DataCap\TradeFinance"</app>
      <app name="ADPDemo" ref="C:\DataCap\ADPDemo"</app>
      <app name="Maintenance" ref="C:\DataCap\Maintenance"</app>
      <app name="Flex" ref="Flex"</app>
      <app name="TravelDocs" ref="TravelDocs"</app>
      <app name="APT" ref="APT"</app>
      <app name="AutomationDocumentProcessing" ref="AutomationDocumentProcessing"</app>
      <app name="FormTemplate" ref="Templates\FormTemplate"</app>
      <app name="LearningTemplate" ref="Templates\LearningTemplate"</app>
      <app name="Transaction" ref="Transaction"</app>
      <app name="watsonxai" ref="watsonxai"</app>
    </datacap>
   

4. Go to the Datacap Application Manager, where the watsonxai application appears in the list of applications.

   

5. Open Datacap Studio, select the watsonx.ai application, and log in using these credentials: admin/admin.

   

6. In the Rulesets section, expand the MyPageID ruleset to view the functions that leverage generative AI to classify a given document. The watsonxai sample application can classify these types of documents: Invoices, Bank Statements, Bills of Lading (BOLs), and Utility Bills.

   

   The sample application uses generative AI to extract metadata from the document, either through predefined keys that are created as attributes in DCO or through attributes identified by the LLM from the OCR text.

   The following three functions are used for classification and extraction generative AI tasks:

   AskAQuestionUsingPageText. Submits the question to the model using a pre-determined prompt layout and automatically provides the text for the current page within the prompt. The action allows a single request to be made with results stored in a single location. The question placed within the prompt could be used to locate single values from within the page text or have the model evaluate the page text for operations, such as classification. The response will depend on the question, the input text and the ability of the model to formulate the desired answer. It is highly recommended that sufficient testing be performed to confirm the answers are acceptable.

   

   AskForPageValuesUsingKeys. The action will dynamically create an LLM prompt using the page text and data from specially configured fields on the current page. The action will use the LLM to dynamically find the values of all of the configured fields. For example, with an invoice, fields can be configured to hold values such as the invoice number, invoice date, invoice total, etc. This action will ask the model to find these fields on the page text, and then automatically populate the associated fields with the values from the text. The text is located using the plain text and does not require fingerprints to be configured that properly identify the location of the text on the page. This action is a good candidate to find values on a page without needing the traditional Datacap setup of a fingerprint that identifies the location of the fields on the page, simplifying the setup of an application.

   

   AskAFreeFormQuestion. This action allows for a complete, user constructed, query to be provided to the LLM, allowing any text to be submitted as-is. Using this action, the user can create a custom prompt for the LLM. The action allows a single request to be made with results stored in a single location. If the returned text exactly matches text on the page, and if the target is a field, then the field will be updated with the coordinates of the found text.

   

Step 4. Run the watsonxai sample application in Datacap Studio

1. In Datacap Studio, go to the Test tab, and run the watsonxai application.

2. Click Demo_SingleTFFs to create a new batch and execute the sample. A popup is displayed, where you click Advance to proceed to next step.

3. In the PageID task, the sample application has been configured with the integration with LLM for document classification and extraction. Review the type of the document ingested, and it shows it is categorized as an “invoice” by the LLM. Also, you can see the fields that are populated using the keys defined in the DCO.

   

Summary

In this tutorial, you learned how to configure watsonx.ai actions into your existing Datacap setup, learned how to add functions that invoke LLMs, and run the sample watsonxai application to see how document classification and template less extraction works in Datacap.

If you have any comments or questions about what I covered in this tutorial, please reach out to me at ela.mishra@in.ibm.com.