Granite models

Introducing Data Prep Kit (DPK) as a streamlined data preparation tool for LLMs

In this tutorial, you learn how to use the Data Prep Kit (DPK) to prepare data for fine-tuning and to achieve the business use case of contract analysis.

In this learning path, learn how to use Data Prep Kit (DPK) to prepare data for large language model (LLM) applications.

In this tutorial, learn how to build a Ray runtime, create unit tests, integrate with CI/CD, and add KFP extensions to existing transforms.

In this tutorial, you learn how to use the Data Prep Kit (DPK) to prepare PDFs for a RAG pipeline.

While Data Prep Kit includes several built-in transforms, you can also build custom data preparation transforms to meet your needs. In this tutorial, learn how to implement a Processing transform that calculates a signature value for a document and stores the signature as part of the metadata associated with the document.

Data Prep Kit (DPK) is a scalable, flexible, robust, and easy to use framework for data processing. Data Prep Kit is data agnostic, handling diverse data formats including text, code, and structured data. It uses distributed computing frameworksvlike Ray and Spark to efficiently process large data sets. Developers can createvcustom transforms (to supplement the built-in transforms) to readily address specificvdata processing needs.

The Data Prep Kit (DPK) offers a diverse set of pre-built transforms to streamline data preparation for AI applications. These transforms cater to various data formats, including text, code, and structured data. These built-in transforms provide a strong foundation for handling common data processing tasks. This rich collection of built-in transforms empowers one to efficiently prepare the data for various AI and machine learning applications.

This tutorial will show you an implementation of Agentic Retrieval-Augmented Generation (RAG). It can perform multi-step workflows like combining document search and web search to perform complex tasks like business research, feature comparison, news retrieval based on projects, personal knowledge management, and more.

Learn how open source impacts watsonx through some of the key open source projects that IBM has invested in.