Iterating Towards LLM Reliability with Evaluation Driven Development

Devin Stein, CEO of Dosu, described in a guest post for LangChain how his team runs evaluation driven development (EDD) and how LangSmith, LangChain's agent engineering platform, lets them scale that practice. Dosu is an AI teammate that helps develop, maintain and support software projects; it launched in late June 2023 and is now installed on thousands of repositories. Dosu is a familiar presence on the LangChain GitHub repository itself, and LangChain was one of Dosu's early partners.
The problem EDD addresses is that LLM products behave probabilistically. Small changes are hard to predict: Stein notes that adjusting a prompt can improve one domain while causing regressions elsewhere. For each change, the team wants to hold performance steady where Dosu is strong and improve it where it struggles, so it needs a way to measure impact rather than eyeball it.
Dosu's workflow follows a repeatable loop: create a new behavior with a handful of initial evals, launch it to users, watch production results to identify failure modes, add examples of each failure mode to the offline eval set, iterate against those evals, then relaunch. In the early days the team inspected every response by hand with grep and print statements. That was slow but valuable, Stein writes, because it taught them how people actually used Dosu and which request types it handled well or badly.
Growth broke the manual approach. With Dosu generating responses around the clock across thousands of repositories and dozens of submodules, monitoring production and spotting failure modes became impractical. Dosu set criteria for a replacement monitoring stack: prompts must live in Git so prompt changes get the same review as code; tracing must work at the code level, tracking metadata across LLM calls within a single trace; data must be easy to export so existing eval datasets and tooling survive; and the tool must be customizable and extensible, since there is no standard way to build LLM apps and the team wanted control over tracked metadata. LangSmith met those requirements.
Instrumentation was light. Dosu added a @traceable decorator to a few LLM-related functions, which took minutes, and traces appeared in the LangSmith UI as soon as the changes hit production. The decorator captures both function and LLM call traces, so raw function inputs, the rendered prompt template and the model output land in one trace. This suggests the appeal is less about dashboards than about joining the non-LLM glue code and the model call in a single record, which is what makes debugging multi-step agents tractable.
To find failures, Dosu watches several signals: explicit thumbs up/down feedback, user sentiment in GitHub issue replies, internal errors such as oversized inputs or outputs or responses that fail a schema, and response time. Dosu prioritizes quality over speed but still wants to know why a slow response is slow, since some requests need fast answers and others need slower, more precise ones. LangSmith's search lets the team filter traces by those signals, and attached metadata extends what can be queried. Among the failure modes this surfaced: extremely slow responses when users paste thousands of lines of logs or raw OpenAI embedding float values, and a case where Dosu, asked to label a pull request, instead told the user about a concert it was excited to attend.
Once a failure mode is identified, the loop repeats: search LangSmith for more examples, add them to eval datasets, iterate against the evals, ship a new version. The team is now customizing LangSmith further to build evaluation datasets automatically from production traffic, so engineers can curate datasets by conversation topic, user segment or request category with minimal effort.
Stein also points to a flywheel between the two companies: LangSmith speeds Dosu's iteration, better Dosu reduces the maintenance and support load on the LangChain team, and more time on LangSmith features in turn speeds Dosu's development. Dosu is hiring.
In practice, the pattern here is a well-known one from software testing applied to nondeterministic systems: production traffic becomes the source of test cases, and each confirmed bad output becomes a permanent regression check. The trade-off is that eval suites grow and need upkeep, and offline evals only approximate live user behavior, so production monitoring has to stay in the loop. Treating prompts as versioned code is the other notable choice, since it makes prompt changes reviewable and revertible like any other diff.
Why it matters: Teams shipping LLM features face the same regression risk Dosu describes, and this account shows a concrete division of labor, with prompts in Git, traces carrying cross-call metadata, and searches over production traces feeding offline evals. Developers evaluating observability tools can read Dosu's four criteria as a useful checklist, particularly easy data export and code-level tracing. The likely implication, as inference rather than reported fact, is that automatic dataset curation from live traffic will become a standard expectation rather than a differentiator.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
Dosu uses evaluation driven development and LangSmith to build reliable LLM products at scale, monitor production performance, and iterate with confidence.