Large language models (LLMs)

Location : Orange County Convention Center, 9400 Universal Blvd., Orlando, 32819, United States

Learn how to evaluate SQL query generation with a robust benchmarking method that handles SQL complexity, semantic diversity, and result validation.

Introducing Data Prep Kit (DPK) as a streamlined data preparation tool for LLMs

In this tutorial, you learn how to use the Data Prep Kit (DPK) to prepare data for fine-tuning and to achieve the business use case of contract analysis.

In this tutorial, learn how to build a Ray runtime, create unit tests, integrate with CI/CD, and add KFP extensions to existing transforms.

Data Prep Kit (DPK) is a scalable, flexible, robust, and easy to use framework for data processing. Data Prep Kit is data agnostic, handling diverse data formats including text, code, and structured data. It uses distributed computing frameworksvlike Ray and Spark to efficiently process large data sets. Developers can createvcustom transforms (to supplement the built-in transforms) to readily address specificvdata processing needs.

While Data Prep Kit includes several built-in transforms, you can also build custom data preparation transforms to meet your needs. In this tutorial, learn how to implement a Processing transform that calculates a signature value for a document and stores the signature as part of the metadata associated with the document.

The Data Prep Kit (DPK) offers a diverse set of pre-built transforms to streamline data preparation for AI applications. These transforms cater to various data formats, including text, code, and structured data. These built-in transforms provide a strong foundation for handling common data processing tasks. This rich collection of built-in transforms empowers one to efficiently prepare the data for various AI and machine learning applications.

In this tutorial, you learn how to use the Data Prep Kit (DPK) to prepare PDFs for a RAG pipeline.

In this learning path, learn how to use Data Prep Kit (DPK) to prepare data for large language model (LLM) applications.