Understanding the AI Agent Technology Stack

The Technology Behind AI Agents

‍AI agents are built different. AI agents have different underlying technology than most traditional software tools in the market. AI agents are **autonomous** and software tools are not, and that is thanks to the different technology driving agents, as well different infrastructure as middleware, but also **language models** are added to the mix to make them intelligent (so they can reason and communicate).

An Introduction to the AI agent stack

The AI agent tech stack

The AI technology stack driving the agent economy can explained with an analogy to the human body, where each component plays a specific role in ensuring the system functions effectively:

Infrastructure: The Legs

Just as legs support the body and facilitate movement, the infrastructure in AI serves as the foundation that supports and enables all AI operations. This includes the hardware for compute power and the necessary data handling capabilities.

Components:

• **Hardware:** Graphics Processing Units (GPUs), Tensor Processing Units (TPUs), and Language Processing Units (LPUs) provide the computational power required to process and analyze large datasets quickly and efficiently.
• **Data:** The vast data pools that feed into AI models for primarily pre-training and some model fine-tuning.
• **LLM Ops:** These are specialized operations tailored to manage and optimize large language models, ensuring they run smoothly and efficiently.

Models: The Brain

What It Is: Representing the brain, AI models are the intelligent core that processes information and makes decisions. These models (like OpenAI's GPT, Anthropic's Claude, Google's Gemini, etc) determine how AI agents interpret and interact with the world.

Components:

• **Language Models:** These models understand and generate language, code and visuals like image and video, enabling agents to communicate effectively.
• **Action Models:** These models are trained to facilitate the execution of specific actions (they 'generate' actions).
• **Fine-tuning:** This process adapts the models to specific tasks or industries, enhancing their accuracy and relevance.

Middleware: The Core

Middleware acts as the core of the system, integrating various components of the AI stack. It orchestrates and distributes tasks and data between the models and the infrastructure.

Components:

• **Orchestration:** This involves managing and coordinating the flow of tasks and data across the system to ensure that everything functions harmoniously (both human to agent or agent to agent).
• **Distribution:** This refers to the promotion, deployment and scaling of AI agents across different environments and users.

Agents: The Hands

Agents are like the hands of the body, interacting with the external world. They apply the intelligence processed by the models to perform specific tasks within their designated roles.

Components:

• **Specialized Agents:** These agents are designed for specific tasks or industries, such as AI legal assistants or AI healthcare workers, where specialized knowledge and capabilities are necessary.
• **General Agents:** More versatile in nature, these agents handle a broad range of activities and automations. They can adapt to various tasks, making them suitable for more general and horizontal responsibilities.

AI agent stack is rapidly developing and improving

The AI agent stack currently is relatively immature still. Technologies across the entire stack, in as well the infra, model, midware and agent layers are rapidly developing to become a lot more efficient, powerful and robust.

Look at the current models for example, these are regarded by their makers as relatively 'dumb' and expected to soon be a lot more intelligent. An example is how **GPT2** was closer to a toddler in terms of output but later models like **GPT4** already outperform professionals on a lot of different performance benchmarks.

Also the other layers of the stack are getting better every day. Processing units get faster exponentially, data gets more rich and synthetic data is generated, LLM ops gets more sophisticated, more and more smaller special purpose finetuned models get trained, orchestration and distribution expand in functionality and scale up, and specialized and general agents get out of proof of concept and deployed at customers as we speak, all compounding to an ever accelerating pace of improvement of AI agents with exponentials stacking up.

Thanks to these exponentials we're going to see a major wave of AI agents being deployed in the coming years.