What Is Eval-Driven Development? How to Improve AI Agents

LangChain has published a walkthrough of eval-driven development (EDD), a process the company says it used while building its most advanced agents and that it reduces to three steps: define what good looks like for the agent, write the eval before building, and compare each change against prior experiments. The post is attributed to the LangChain team and framed around LangSmith, its agent engineering platform, but the core method is framework-agnostic.
The pitch rests on a familiar failure mode. Early in an agent's life you write a prompt, try a handful of questions by hand, tweak, and repeat; errors are obvious and improvements easy to see. Over time it becomes hard to tell whether the latest change made the agent better or worse. EDD replaces ad hoc spot checks with a fixed set of metrics that guide decisions, so each iteration can be judged against the same yardstick.
The name and concept borrow from test-driven development, where you write a failing test before the code that passes it. Agents complicate this because they are non-deterministic: the same question can get different answers on different runs, more than one natural-language answer can be correct, and the agent chooses its own steps, sometimes calling an unnecessary tool or skipping a needed one while still landing on the right answer. An eval handles this by scoring runs across a set of examples rather than asserting a single expected output. Evals can score almost any aspect of a run — the final answer, an intermediate step, the full sequence of tool calls, or a multi-turn conversation grouped into a thread — and can use another LLM as a judge where a code check is too brittle.
Step one is to define success. Start from what the agent should accomplish, then list the behaviors needed to get there, such as answering from a knowledge base, acting on a user's behalf, or knowing when to ask a follow-up. Together these form a product requirements document for the agent. The post stresses working on one behavior at a time so a narrow target can be hill-climbed without one change muddying the effect of another. Before writing code, write a short spec covering when the behavior should fire, what the agent should do, what it should not do, and which existing behaviors must keep working. The spec needs to be definitive enough to score pass or fail.
The worked example is an internal docs agent with two tools, search_docs for internal documentation and web_search for everything else, to which the team plans to add a create_ticket tool for IT tickets. The spec: when a user reports something broken, like a laptop that will not connect to the VPN, call create_ticket; do not file a ticket for how-to questions such as how to connect to the VPN; and keep search_docs and web_search handling what they handle today. The eval is written against these guidelines even though the capability does not exist yet.
Step two is writing the eval. A dataset is a collection of examples with inputs and, optionally, reference golden outputs. Real agent runs are one of the best input sources, because users phrase things unpredictably and surface edge cases you would not invent; the closer examples are to real traffic, the more a good score means good production behavior. In LangSmith, real runs can be added to a dataset directly. Datasets can also be extended synthetically — for instance, rewriting existing examples as angrier messages to check the agent still picks the right tool. For the docs agent, a handful of examples suffice: ticket requests plus questions that should still route to the two search tools.
The evaluator turns the definition of good into a score. Deterministic checks belong in code; subjective ones belong to an LLM judge. For the docs agent, tool choice is easy to verify in code, so each example passes if the agent calls the tool listed in its reference output. The post notes that more complex agents need more than spot-checking tool calls and the final response. An agent that changes its environment, like a coding agent in a sandbox, needs its end state graded, and each run needs a clean environment so runs do not interfere. Harbor, an open source framework and the harness behind Terminal-Bench, runs each task in its own sandbox with a test script grading the end state. LangChain says it built its own benchmark from real and synthetic runs for LangSmith Engine and uses Harbor to benchmark Deep Agents, checking whether parts of the harness — the todo-list middleware, parts of the system prompt — can be cut without hurting performance.
Before changing anything, run a baseline: score the current agent on every example and log it as an experiment in which each example's input, output and scores are saved together. With the docs agent, ticket examples fail because the tool does not exist yet, while the rest record existing behavior so later regressions are detectable.
Step three is making the change and comparing. After building create_ticket, the same eval runs again as its own experiment, compared side by side with the previous one to show which examples improved and which regressed. In the example, ticket cases now pass, but asking how to connect to the VPN files a ticket instead of searching the docs: the average score rises while one case breaks. Reviewing the failing trace — the record of a run including every model and tool call — reveals the tool description says to use create_ticket for VPN and laptop issues, so any VPN question matches. Narrowing the description to requests that need IT to act gets every example passing. The post warns that a perfect score on a small dataset is a good sign but not always the target, since tuning against the same examples risks overfitting; a held-out set you do not inspect while iterating reveals it when scores rise on tuned examples but stay flat on held-out ones.
Keeping EDD running means automating it. Code checks fast enough to block a bad merge can gate pull requests, while slower LLM-judge evals run nightly or before a release; dataset splits let you run only the relevant behavior while iterating and the full set before shipping. LangSmith integrates with pytest and Vitest, logging every run as an experiment. In production, online evaluators score live traces on a sampled share of traffic to manage cost, annotation queues let teams review flagged runs by hand and validate that a judge agrees with people, dashboards track eval scores, feedback, cost and latency over time, and Insights clusters traces to surface new patterns. LangSmith Engine goes further, turning clusters of issues into dataset examples with assertions for review. LangChain also publishes the evals it uses for Deep Agents in that project's repository as a real production suite.
The post closes with common mistakes: defining success too loosely ("the response is helpful" is not a well-defined metric), writing the eval after seeing the fix so it passes by design, watching only the average score, adding evals you do not need, and trusting an unchecked LLM judge instead of validating it against human scores with narrow pass/fail questions rather than ambiguous 1-to-10 scales.
Why it matters: teams shipping agents tend to discover regressions in production, where failures are costly and hard to attribute. Adopting an eval-first loop, held-out examples and CI gating gives developers objective evidence that a prompt or tool change helped, and the same discipline applies to product decisions about when an agent is ready to ship. My inference: the main cost is upfront effort in building datasets and judges, which explains why the tooling increasingly aims at harvesting production traces automatically.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
Eval-driven development helps you build AI agents against a consistent set of metrics. Define good behavior, write the eval first, then compare experiments.