Thoughts on Agent Frameworks

AI agents are all the rage nowadays, and developers are jumping into this fray at fervent speed. There are a number of frameworks one can use, such as LangChain/LangGraph, AutoGen from Microsoft, CrewAI, OpenAI Agents SDK, Google’s Agent Development Kit, etc, and as I learn more about them, here’re my quick thoughts on them.

First of all, one doesn’t have to use any frameworks to develop agents. You can write AI agents using any programming language from scratch, and it is not that difficult actually. All agents need to include several types of components:

1. LLMs to process inputs and make decisions on what responses to give. This can be simply an API call to one of the LLMs. For example, one can use LiteLLM as a gateway to hundreds of different LLMs.
2. Tools to call to make changes. Unlike chatbots, agents need to take up actions, such as responding to an email. A developer needs to have tools available for the LLMs to invoke to take on these actions, and the LLM needs to know what tools are available, what they do and how to invoke them. Tools can be just plain functions or wrappers of external tools.
3. Memory. An agent needs to understand what has been done before, and a memory or state object is used to capture and pass from agent to agent.

Although all of these are easily doable on one’s own, there are a few benefits by using an existing framework:

1. Battle tested. Because a framework has been used by many people, they are tested more thoroughly than one’s own implementation.
2. Ecosystem of libraries and utilities. For each major framework, there is an ecosystem of libraries and utilities. For example, there are many visualization, monitoring and alerting libraries for each framework. This makes development and production deployment much easier.
3. A community of users. If you run into problems, you can find a community of users that can

The downside of frameworks are:

1. It takes a bit of time to learn them. Each framework has its own idiosyncrasies, and requires a little bit of learning curve.
2. You may be locked into a framework/ecosystem. The OpenAI Agents SDK is heavily integrated with the OpenAI ecosystem, and Google’s Agent Development Kit (ADK) is heavily integrated with the Google ecosystem, particularly Google Cloud and Gemini models.