LLM Powered Autonomous Agents

Lilian Weng published an overview of LLM-powered autonomous agent systems, describing a design in which the large language model acts as the agent's brain alongside several key components.

The concept of building agents with an LLM as core controller is described as promising, with proof-of-concept demos such as AutoGPT, GPT-Engineer and BabyAGI cited as examples. The potential of LLMs is framed as extending beyond generating text and programs to serving as a general problem solver.

The overview outlines three components. Planning covers subgoal decomposition, in which an agent breaks large tasks into smaller manageable subgoals, and reflection and refinement, in which an agent performs self-criticism and self-reflection over past actions to learn from mistakes and improve future steps. Memory covers short-term memory, which the overview equates with in-context learning, and long-term memory, described as retaining and recalling information over extended periods, often through an external vector store and fast retrieval. Tool use covers learning to call external APIs for information missing from model weights, including current information, code execution, access to proprietary sources and more.

For planning, the overview describes task decomposition approaches including chain of thought, in which a model is instructed to think step by step to decompose hard tasks; Tree of Thoughts, which explores multiple reasoning possibilities per step in a tree structure searched by BFS or DFS; and LLM+P, which uses an external classical planner via Planning Domain Definition Language as an intermediate interface.

On self-reflection, it cites ReAct, which integrates reasoning and acting by combining task-specific discrete actions with the language space, and Reflexion, which equips agents with dynamic memory and self-reflection. Chain of Hindsight presents a sequence of past outputs annotated with feedback, while Algorithm Distillation applies the idea to cross-episode trajectories in reinforcement learning.

For memory, the overview discusses maximum inner product search and approximate nearest neighbors algorithms including LSH, ANNOY, HNSW, FAISS and ScaNN. For tool use, it cites MRKL, TALM, Toolformer, ChatGPT Plugins and OpenAI API function calling.