Get to know AI agents

Imagine that you have a friend who's good at finding the best pizza places in town. You can tell your friend what you like in a pizza, and they'll use that info to suggest the perfect place. AI agents work kind of like that, but instead of a friend, it's a computer program that learns from lots of examples to help us with all sorts of things, from recommending movies to driving cars autonomously.

Architecture of an AI agent

An AI agent is a wrapped large language model (LLM) with extra abilities to comprehend its environment and take actions upon. AI agents learn from observing the environment.

At its core, an AI agent is made up of these components:

• Environment. The area or domain (knowledge) in which an AI agent operates.
• Sensors. AI agents use sensors to perceive its environment such as cameras or microphones.
• Actuators. These are the tools with which an AI agent interacts with its environment.
• Decision-making mechanism. This reasoning is the brain of an AI agent that is powered by an LLM that describes how the data collected is translated into actions that support the agent’s objective.
• Learning System. This system enables the AI agent to learn from its experiences and interactions with the environment using reward-based machine learning, or reinforcement learning.

How AI agents work

An LLM-powered AI agent’s workflow looks like this:

• Perceiving the environment. The AI agent receives input from the user or the environment.
• Processing input data. The input data is processed to understand the context and requirements.
• Decision making. Based on the processed data, the AI decides on the necessary actions.
• Planning and executing an action. The AI plans and executes the tasks needed to achieve the objective.
• Learning and improvement. The AI continuously learns from the results and feedback to improve its performance.

Example of an AI agent, a smart home assistant

Let’s walk through a real life example of a smart home assistant:

1. Perceiving the environment:

   • The AI agent receives input from the user: "Turn on the living room lights."

2. Processing input data:

   • The AI processes the input to understand that the user wants the living room lights turned on.

3. Decision making:

   • Based on the processed data, the AI decides that it needs to send a command to the smart lighting system.

4. Planning and executing an action:

   • The AI plans the sequence of actions needed: communicate with the smart lighting system, locate the living room lights, and send the command to turn them on.
     • The AI executes the plan by sending the command to turn on the living room lights.

5. Learning and improvement:

   • The AI learns from this interaction, noting the user's preferences for light settings and the specific wording used.
   • It uses this feedback to improve future interactions, making the process faster and more accurate.

This example illustrates a simple interaction with a smart home assistant, showing how the AI perceives, processes, decides, plans, executes, and learns from the task of turning on the lights. This structured approach ensures that the AI agent operates efficiently, prioritizing and executing tasks systematically while learning and improving over time.

Autonomous AI agent workflow

Let’s walk through our smart home assistant through this detailed workflow, using the IBM’s Granite LLM, as illustrated in the following image.

1. Provide objective and task:

   • The user provides the initial objective and task: "Turn on the living room lights."

2. Execution agent:

   • The AI agent fetches the context by querying memory. The AI agent queries its memory to understand the current state of the living room lights and any relevant user preferences.

3. Task creation agent:

   • Based on the context, the AI agent generates any necessary ancillary tasks. The AI agent determines that it needs to identify the specific smart light devices in the living room and establish a connection to them.

4. Add new tasks:

   • The task creation agent adds the newly generated tasks to the memory: The AI logs the tasks of identifying and connecting to the living room lights into its memory.

5. Task prioritization agent:

   • The task prioritization agent prioritizes the tasks based on their importance and urgency. The AI agent prioritizes the task of turning on the lights immediately over other non-urgent tasks.
   • The prioritized task list is sent back to memory for storage.

6. Execution agent:

   • The execution agent queries memory for context, retrieves the tasks, and completes them. The AI agent retrieves the prioritized task of turning on the lights, identifies the correct devices, and sends the command to turn on the lights.

7. Send task result:

   • The execution agent sends the task results back to memory. The AI agent confirms that the lights have been turned on and updates the memory with this result.

8. Store task/result pair:

   • The task and its result are stored in memory for future reference. The AI agent stores the user command (task) and the successful activation of the lights (result) in memory, so it can use this information for future interactions.

The next big thing

AI agents have been here for a long time already however with the rapid increase in Gen AI & LLM capabilities. It is now easier than ever to design and develop an end-to-end AI agent. DEVIN is one such example which is said to be able to work as a human software engineer. The self-driving car company WAYMO is another example that uses this technology.

Summary and next steps

In this article, you learned how AI agents work, explored how they perceive their environment, process data, make decisions, and execute actions, all while continuously learning and improving.

From turning on your living room lights to potentially driving your car, AI agents are becoming an integral part of our daily lives. Their ability to learn and adapt makes them not just tools, but dynamic helpers ready to make our lives easier and more efficient.

The future with AI agents is not just exciting—it's already happening!

Dive deeper in this article that defines AI agents.