New in LangSmith Engine: red teaming and automated testing

LangChain announced LangSmith Engine v2 on September 24, 2026, adding Red Teaming and automated testing to the in-platform agent that automates steps of the agent development lifecycle. The announcement was written by Ben Tannyhill and Trammell Saltzgaber.

Red Teaming is described as a proactive troubleshooting tool that catches issues such as hallucinations and violations of system prompts before they affect users in production. According to LangChain, it analyzes an agent's production traces and repos to understand its purpose and behavior, then uses that context to test for weaknesses and identify issues that have not yet surfaced in production. Red Teaming is available today in Private Beta to existing LangSmith Deployment users, with an application process to join.

Engine v2 also adds detection for harder-to-spot issues. It analyzes agent trajectories to determine whether an agent is taking an inefficient path, such as an incorrect or repetitive tool call, and proposes a more efficient alternative. It tracks trends in error rate, latency, and cost. LangChain said each issue is tracked in the same queue so users have one place to track and resolve agent issues.

For agents using LangSmith Deployment, Engine now validates proposed fixes before placing them in the review queue. When it spots an issue, it first reproduces the failure by running the offending inputs against the agent in Deployment, then proposes a change, tests it against the same inputs, evaluates results, and adjusts the fix. Once Engine confirms the fix resolves the issue, it provides the fix in LangSmith, where users can open a PR with a click. This feature is in Private Beta today for existing LangSmith Deployment users.

LangChain said Engine has analyzed more than 70 million traces and diagnosed tens of thousands of issues since launching in May. It also cited August figures: Engine is more than 2x better at detecting issues as measured by IssueBench and provides fixes that are 25% more effective as measured by Terminal-Bench.

Engine v2 is available in SaaS deployments for all LangSmith Plus and Enterprise plans. Support for Engine v2 in Self-Hosted plans is coming shortly, and the next self-hosted LangSmith release will support bring-your-own-key for Engine, allowing teams to provide their own model API key for inference while trace data stays within their VPC.