IBM Developer

Article

IBM watsonx.ai on AWS

New enterprise studio for generative AI and machine learning builders on Amazon Web Services

By Judah Bernstein, Sunil Gajula

On this page

Generative AI is democratizing artificial intelligence (AI). The proliferation of foundation and generative pre-trained transformer (GPT) models is accelerating our clients' ability to realize the potential of AI-first business applications.

Foundation models are self-supervised machine learning models trained on a vast quantity of data at scale. These models can be adapted or tuned for distinct business applications across a broad set of use cases. Clients are leveraging foundation models for a wide range of purposes, including:

  • Content summarization
  • Extraction
  • Classification
  • Content generation (including text, images, video, code, or molecules)
  • Sophisticated question answering
  • Software code generation.

Foundation models allow clients to accelerate value by enabling inference on large pre-trained models. Traditionally, customers would not have been able to train and tune those models themselves because of exorbitant costs, inability to access infrastructure capacity, and complexity of sourcing, collecting, and aggregating data. Foundation models often provide general-purpose models that expedite a client’s ability to leverage machine and deep learning within their organizations. An added benefit is the ability to fine-tune the model for a prescriptive use case, often with increased accuracy.

Available from a broad set of model providers, foundation models might come from independent software vendors (ISV), educational institutions, cloud providers, or open source developers. They are often externally available through model hubs such as Hugging Face. Examples of foundation models include large language models (LLM), IT automation, digital labor, cybersecurity, and others.

Organizations are accelerating their adoption and use of foundation models. As they ramp up, they are looking to use open source frameworks and tools that bring AI builders together, tune their foundation models for their business with minimal data and advanced prompt-tuning capabilities, and manage the AI lifecycle in one place to train, validate, tune, and deploy AI models.

New enterprise studio for generative AI and machine learning

Earlier this year at IBM Think 2023, IBM announced watsonx.ai, our enterprise studio for generative AI and machine learning builders. Watsonx.ai is an enterprise-ready studio for AI builders, bringing together traditional machine learning and new generative AI capabilities powered by foundation models.

IBM watsonx.ai home page

Key watsonx.ai features include:

  • Foundation models | Clients have access to a family of IBM-trained foundation models of different sizes and architectures, a selection of open source models from Hugging Face, and third-party models, including Llama-2-chat and StarCoder LLM for code generation. IBM models include Slate, for non-generative AI tasks, and the Granite series models, which use a decoder architecture to support a variety of enterprise Natural Language Processing (NLP) generative AI tasks.

  • Prompt Lab | AI builders can work with foundation models and build prompts using prompt engineering. Within the Prompt Lab, users can experiment with zero-shot, one-shot, or few-shot prompting to support a range of NLP-type tasks, including question answering, content generation, summarization, text classification, and extraction.

  • Tuning Studio | Subsequent watsonx.ai releases will include capabilities for prompt tuning and fine-tuning foundation models. The Tuning Studio will help you tune your foundation models with labeled data for better performance and accuracy.

  • Data science and MLOps | All the tools, pipelines, and runtimes a data scientist or application developer needs to build machine learning models and applications, both programmatically and visually. Automate the entire AI model lifecycle, from development to deployment, with connections to a variety of APIs, SDKs, and libraries.

We have seen significant adoption of watsonx over the last few months, and the excitement from our client and partner community has been incredible. Through our dialogue with this community, we've received a significant amount of interest across hybrid cloud environments, including in the public cloud. Working collaboratively with AWS on our strategic collaboration, earlier this year at the AWS New York City Summit we were excited to announce and launch IBM watsonx.data on AWS, our next-generation data lakehouse on AWS.

IBM watsonx.ai certified on AWS

Today, at AWS re:Invent 2023, we are excited to inform our customers that watsonx.ai is certified to run on Amazon Web Services. Watsonx.ai is now available to purchase through IBM and our Business Partners as customer-managed software on Red Hat OpenShift Container Platform (OCP) on AWS IaaS. In 2024, we have planned availability for IBM watsonx.ai on Red Hat OpenShift Service on AWS (ROSA) and the AWS Marketplace.*

Watsonx.ai provides a diverse set of open source and IBM-built foundation models to support a wide variety of use cases, and we are continuing to add models to ensure our customers can achieve their goals for hybrid cloud, multi-modal, open, and governed generative AI within their AWS environments.

Recognizing that a single model won't fit the unique needs of every business use case, Granite models are being developed in different sizes. These IBM models, built on a decoder-only architecture, aim to help businesses scale AI.

For example, businesses can use the models to:

  • Apply retrieval-augmented generation for searching enterprise knowledge bases to generate tailored responses to customer inquiries
  • Use summarization to condense long-form content, such as contracts or call transcripts, into short descriptions
  • Deploy insight extraction and classification to determine factors like customer sentiment

IBM has invested heavily for years in developing its foundation models. They are custom-built for the specific requirements of business use cases, which by definition are much more focused than consumer AI and demand the highest levels of precision and accuracy. IBM's internal benchmarking shows that specialized models can deliver better accuracy, and do so with lower infrastructure requirements, to achieve better price performance.

IBM believes in the creation, deployment, and utilization of AI models that advance business innovation responsibly. That's why the company is also announcing that its standard intellectual property protection—similar to what it provides for hardware and software products—will apply to IBM-developed watsonx models.

IBM provides an IP indemnity (contractual protection) for its foundation models, enabling its clients to be more confident AI creators when using their data, which is the source of competitive advantage in generative AI. Clients can develop AI applications using their own data along with the client protections, accuracy, and trust afforded by IBM foundation models.

The following image shows a selection of the foundation models currently available in the watsonx.ai library.

IBM watsonx.ai foundation model library Figure 1: watsonx.ai foundation model library

Architecture on AWS

Clients interested in watsonx.ai as customer-managed software deployed on Red Hat OpenShift Cloud Platform (OCP) can obtain access by purchasing the solution directly from their IBM seller or Business Partner. After doing so, they obtain access to their entitlement, which allows them to deploy the products within the watsonx.ai platform on AWS. Once they have the entitlement, they can follow the deployment steps required to install the software on Red Hat OpenShift Cloud Platform, and then instantiate an instance of the application within the user interface to access the highly differentiated capabilities.

IBM watsonx.ai reference architecture on AWS Figure 2: IBM watsonx.ai reference architecture on AWS

Watsonx.ai is powered by Amazon Elastic Compute Cloud (EC2) P4 instances, with IBM support for the p4de.24xlarge instance, which includes 8 NVIDIA A100 80GB Accelerators for an aggregate of 640GB HBM2e of memory, 400Gbps ENA and EFA network bandwidth. This top-tier performance can ensure you can obtain a performant, low-latency, and strong concurrency for generative AI prompting, including for large language models in our library, such as Meta’s Llama-2 70B. As you need to increase your concurrency for prompts per second, you can continue to add capacity for these instance types to enable high availability, and scale up your required capacity.

Self-managed Red Hat Openshift Cluster architecture on AWS Figure 3: IBM watsonx.ai deployment architecture on Red Hat OpenShift Cloud Platform (OCP) on AWS

Use cases to get started with watsonx.ai on AWS

Start experimenting with foundation models and build prompts for various use cases and tasks. You can save and reuse older prompts, use different models from our library, and experiment with zero-shot, one-shot, or few-shot prompting to get the best results.

1. Knowledge management

Build a Q&A resource from a broad internal or external knowledge base across your company’s proprietary data to share organizational insights. With the help of AI tasks such as retrieval-augmented generation (RAG), you can analyze multiple documents and data inputs, provide effective responses based on real-time information feeds, and improve documentation quality. RAG is a pattern that's used often because of its unique benefits for enterprise organizations; however, it occasionally introduces accuracy flaws from a model’s response. An effective method to enhance the accuracy of generated content is to incorporate necessary facts as context within the prompt text. This contextual information guides the model towards producing more precise and factually grounded outputs.

You can expand this method of integrating context into prompts by leveraging information from a knowledge base. The retrieval-augmented generation pattern comprises three fundamental steps:

  1. Search for relevant content in your knowledge base: Identify and retrieve pertinent information from the knowledge base that aligns with the desired context.
  2. Pull the most relevant content into your prompt as context: Integrate the extracted relevant content into the prompt text to provide context for the model.
  3. Send the combined prompt text to the model to generate output: Present the amalgamated prompt text, encompassing both user-provided input and the context from the knowledge base, to the model for producing accurate and contextually informed output.

This tutorial incorporates a sample notebook that demonstrates the utilization of the retrieval-augmented generation pattern method. The tutorial aims to showcase how this method can enhance the accuracy of the generated output by providing step-by-step guidance within the notebook environment.

Analyze large amounts of data to identify and extract insights or facts from documents or reports, customer interactions, security, or IT incidents. Discover patterns, trends, or anomalies that occur within the data with the use of generative AI. From the themes extracted, traditional AI and machine learning algorithms can then forecast and make predictions, such as calculating credit risk, future sales, demand forecasts, and inventory management, specific to the user's needs and business requirements.  This tutorial walks you through extracted details from a complaint.

3. Generate synthetic tabular data

Generate synthetic tabular data to protect sensitive information during testing stages. Address data gaps and reduce the risk of exposing an individual’s personal data by taking advantage of data created artificially through computer simulation or algorithms. Further build and test AI and ML models with synthetic data to overcome data gaps and, in turn, improve speed to market with new AI solutions.

This tutorial walks you through the steps of utilizing the Synthetic Data Generator's graphical flow editor tool. It illustrates the process of generating synthetic tabular data using either production data or a custom data schema. The tutorial utilizes visual flows and modeling algorithms, offering a user-friendly approach to creating synthetic data sets.

4. Generate content, technology, and code

You can create new technology, content, and code through the power of generative AI to support developer and business user productivity across a range of business domains. The content generated can include lesson planning and curriculum development, ideas for marketing and sales campaigns, emails, blogs, social media posts, product demonstrations, synthetic data images, technical documentation, user persona development, automated reports, scripts, and more. This tutorial walks you through the steps to create code from a set of instructions.

Summary and next steps

In this article, we've reviewed the considerable value that businesses are extracting from generative AI and foundation models. We highlighted the certification of IBM watsonx.ai on AWS as customer-managed software on Red Hat OpenShift Cloud Platform on AWS, and looked at our strategic roadmap to bring watsonx.ai to Red Hat OpenShift Service on AWS (ROSA) and AWS Marketplace next year. We walked through watsonx.ai features, the foundation model library, and use cases that our clients are leveraging on the AWS Cloud.

Are you attending AWS re:Invent 2023? Stop by the IBM booth, #930, at the Expo Hall. And don’t miss these sessions from IBM experts exploring hybrid cloud and AI:

  • Hybrid by Design at USAA: 5:00 p.m., Tuesday, November 28, The Venetian, Murano 3306
  • Scale and Accelerate the Impact of Generative AI with watsonx: 4:30 p.m., Wednesday, November 29, Wynn Las Vegas, Cristal 7

Ready to get started? Get more information on products, capabilities, and the IBM / AWS partnership:

IBM | AWS partnership

With Amazon Web Services (AWS) and IBM, unleash the transformative value of generative AI in your business with greater speed, scale and trust. The IBM, AWS, and Red Hat partnership brings a unique combination of leading enterprise AI, cloud, infrastructure, and open source technologies delivered with deep IBM consulting expertise. This enables companies to quickly and responsibly scale AI workloads using a comprehensive stack of generative AI, composed of Amazon Bedrock and IBM watsonx running on AWS Cloud and across hybrid cloud environments.

Learn more about the IBM / AWS partnership.

*IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.