Recent developments in neural networks and language models have opened the door to a world with new levels of automation, enabling the programmatic generation of text and images that appear human-written. Projects like OpenAI have provided impressive examples of stand-alone services that use this technology, such as DALL-E 2 and ChatGPT.

IBM API Connect Essentials (formerly StepZen) is taking this a step further and enabling you to integrate this technology with existing APIs and services. This tutorial shows how to add GPT-3 text and image generation into an existing GraphQL API with API Connect Essentials.

You can find the complete code for this tutorial in the GitHub examples repository. And, you can watch the video walkthrough on our YouTube channel.

Video will open in new tab or window

Notice: StepZen was acquired by IBM in 2023 and renamed to API Connect Essentials. Some artifacts still use the StepZen name.

Introduction

You can start from a GraphQL API for the RSS feed. This API fetches the last N posts, including the title, URL, full text, and metadata. Then, you extend it by adding a pair of OpenAI-powered properties to each blog post entry: summary and image. The summary property contains a GPT3-generated short summary of the post, and the image property links to a DALL-E 2-generated cover image.

Ultimately, you have a GraphQL API that lets you fetch posts from an RSS feed and augments them using a GPT-3-based language model and the DALL-E 2 image generation network.

DALL-E 2 image

To add a bit more fun to this example, I tasked the language model to create each summary as a limerick. For example:

stepzen request '{ feed(limit: 1) { title guid summary {choices{text}} }}'
{
  "data": {
    "feed": [
      {
        "title": "How to Build a Headless CMS using Notion and GraphQL",
        "guid": "https://stepzen.com/blog/how-to-build-a-headless-cms-using-notion-and-graphql",
        "summary": {
          "choices": [
            {
              "text": "A Notion project was in sight
                       So he decided to give it a try
                       He needed an API to make it work
                       And he found GraphQL to be a perk
                       With a GraphQL API, now his blog site will fly"
            }
          ]
        }
      }
    ]
  }
}

The first step is to import the OpenAI text completion API into your GraphQL schema, where you can combine it with the other APIs.

Step 1. Create a GraphQL API for OpenAI text completion

OpenAI provides a REST API to execute text completion tasks on their GPT-3-based language models. To use this API, you must first sign up with OpenAI and get a personal API key. After you have an OpenAI API key, you can use the stepzen import curl command from the API Connect Essentials CLI to automatically create a GraphQL schema for this API.

stepzen import curl https://api.openai.com/v1/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
      "model": "gpt-3.5-turbo",
      "prompt": "What is a language model and how is it related to AI?",
      "temperature": 0.7,
      "max_tokens": 256,
      "top_p": 1,
      "frequency_penalty": 0,
      "presence_penalty": 0
    }' \
  --name openai \
  --query-name textCompletion \
  --query-type TextCompletion

You can manually adjust the auto-generated textCompletion GraphQL query field in the generated schema file to tailor it to your needs. For example, you might want to reduce the number of required arguments by adding default values.

type Query {
      textCompletion(
        prompt: String
        frequency_penalty: Float = 0
        max_tokens: Int = 256
        model: String = "gpt-3.5-turbo"
        presence_penalty: Float = 0
        temperature: Float = 0.7
        top_p: Float = 1
      ): TextCompletion
        @rest(
          method: POST
          endpoint: "https://api.openai.com/v1/completions"
          headers: [{ name: "authorization", value: "$authorization_8bfcd33539;" }]
          configuration: "curl_import_config"
        )
}

At this point, you have added a textCompletion query field to your GraphQL API. You can check that it works by deploying your GraphQL schema with stepzen deploy and then sending requests. For example:

stepzen request '{
  textCompletion(prompt: "What is a language model?") {
    choices { text }
  }
}'

The next step is to import the OpenAI image generation API to your GraphQL schema.

Step 2. Create a GraphQL API for DALL-E image generation

Similar to the text completion OpenAI features, the DALL-E image generation is available through a REST API. The command to import it into an API Connect Essentials workspace looks like the following example.

stepzen import curl https://api.openai.com/v1/images/generations \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "prompt": "A cute baby sea otter"
  }' \
  --name openai \
  --query-name generations \
  --query-type GenerationResult

This command creates a GraphQL schema that you can modify manually to fit best into the GraphQL schema that you already have.

In this example, you rename the auto-generated files to keep the image generation and text completion queries organized inside the same openai folder. You can see the exact result in the with-openai folder in the API Connect Essentials examples repository on GitHub.

With minimal effort, you now have a GraphQL API that can be used to access OpenAI’s images/generations API. You can try it out with the following code.

stepzen request '{
  generations(prompt: "a sail flying over the horizon into the sunset") {
    created data { url }
  }
}'

The next step is to combine the new textCompletion and generations queries with the existing GraphQL API for the blog posts feed.

Step 3. Adding OpenAI-based properties to an existing GraphQL type

The feed query field in the blog GraphQL API returns a list of BlogPost objects. Each BlogPost includes the full text of the blog entry (as HTML) in the encoded property.

type BlogPost {
  guid: ID
  title: String
  description: String
  encoded: String
  link: String
  pubDate: String
}

The goal is to extend this type with summary and image properties. With API Connect Essentials, you can do this by creating an extension type with the new properties and by using the @materializer directive to populate them with the results from other queries.

extend type BlogPost {
  summary: TextCompletion
    @materializer(
      query: "summary"
      arguments: [{ name: "fulltext", field: "plaintext" }]
    )
  image: GenerationResult
    @materializer(
      query: "image"
      arguments: [{ name: "fulltext", field: "plaintext" }]
    )
}

Now, the remaining part is to define the summary(fulltext) and image(fulltext) query fields.

Add a `summary(fulltext)` query field

The summary field is calculated in three steps:

Convert the HTML-encoded blog post content into plain text.
Prepare the exact text prompt for the OpenAI text completion API.
Call the OpenAI text completion API with the prepared text prompt.

Steps 1 and 2 are done with small JavaScript snippets that are powered by the ecmascript property of the @rest custom directive, and the first step is materialized into a plaintext property on the extended BlogPost type for convenience.

Then, steps 2 and 3 are executed in a sequence by using the @sequence custom directive.

See the example source code for details.

You can verify that this step works by deploying the updated schema to API Connect Essentials, and then sending a request to the summary query field.

stepzen request '{
  summary(fulltext: "This is a very long and insightful text.") {
    choices {
      text
    }
  }
}'

Add an `image(fulltext)` query field

The image field is defined similarly to the summary field but invokes the OpenAI APIs twice. First, it uses the text completion API to generate a text prompt for the image generation service, and then it calls the image generation service with the generated prompt. The full steps sequence looks like the following steps.

Convert the HTML-encoded blog post content into plain text.
Prepare the exact text prompt for the OpenAI text completion API to generate an image description based on the blog post's full text.
Call the OpenAI text completion API with the prepared text prompt and get an image description.
Prepare the exact text prompt for the OpenAI image generation API.
Call the OpenAI text completion API with the prepared text prompt and get an image description.

Step 1 reuses the plain text materialized into the plaintext property.

Steps 2 and 4 are done with small JavaScript snippets that are powered by the ecmascript property of the @rest directive.

All of the steps are executed in a sequence using the @sequence custom directive.

See the example source code for details.

You can verify that this step works by deploying the updated schema to API Connect Essentials, and then sending a request to the image query field.

stepzen request '{
  image(fulltext: "This is a very long and insightful text.") {
    data {
      url
    }
  }
}'

Putting it all together

You’ve added two new properties to the BlogPost type, and now the clients of your GraphQL API can also fetch a summary and a cover image for every blog post with a query like the following code example.

stepzen request '{
  feed(limit: 2) {
    title
    guid
    pubDate
    summary {
      choices {
        text
      }
    }
    image {
      data {
        url
      }
    }
  }
}'

API Connect Essentials allows federating and combining different APIs in a single GraphQL schema. This is a powerful concept, and it allows extending the APIs that you have no control over with additional capabilities, such as adding GPT-3-based text completion, DALL-E-2-based image generation, and many others.

Users of the GraphQL API layer see a single combined API without the need to orchestrate multiple back-end calls, combine and convert data, and think of the operational aspects of API hosting, such as deployments, releases, and maintenance.

This might be useful for data ingestion API clients because they can use this API to augment and enrich the data that they fetch.

However, this is not a universal solution. Exposing this API to read-intensive clients (such as web pages) is a bad idea. Generating a new unique summary and cover image on each read operation is unnecessarily expensive and slow. This example would be a better fit for APIs where reads fetching the same data are infrequent (such as data pipelines and ETL operations).

Conclusion

In this tutorial, you used API Connect Essentials to generate both text and images from existing content using large language models (LLM)s. For this tutorial, you used OpenAI, but the same is possible with other LLM platforms. Looking to build more complex use cases using Generative AI, LLMs and GraphQL? Have a look at watsonx Flows Engine, which allows you to build AI flows for Retrieval Augmented Generation (RAG) and Tool Calling using GraphQL.

We'd love to hear your feedback on this tutorial, on GraphQL, on REST, or on API Connect Essentials. Questions? Comments? Join our Discord community and let us know.

Creating GraphQL APIs for OpenAI text and image processing

Creating GraphQL APIs for OpenAI text and image processing