Optimizing Agent Reliability and Tool Calling Efficiency for Chip Design Using NVIDIA NeMo Gym

NVIDIA published a developer blog post describing how its NeMo Gym library was used with Siemens EDA to evaluate and improve an AI agent that automates standard-cell characterization in chip design. The Siemens Fuse team used NeMo Gym to scale evaluations of the Siemens Fuse EDA AI Agent's Liberty file generation, raised the measured benchmark composite score from approximately 55% to 95%, and cut agent turns from 155 to 53 while output tokens dropped from roughly 81K to 29K. NeMo Gym is described as an open library for evaluating and improving models and agents using environments, and it is part of the NVIDIA NeMo collection of open libraries for agent specialization and optimization.
Standard-cell characterization generates timing, power, and noise models for logic cells across operating conditions. It is foundational to chip design and is required whenever a semiconductor foundry issues a new process design kit. A PDK carries SPICE transistor models, design rules, and process, voltage, and temperature definitions that connect manufacturing to design tools. When a PDK is updated, existing characterization datasets may no longer describe the updated devices, so timing and power models must be regenerated and validated for every affected cell across combinations of input slew, output load, and PVT conditions. The Solido Characterization Suite agent orchestrates tools from the broader suite through the Siemens Fuse EDA AI Agent, which exposes MCP servers and skills to coding harnesses including Claude Code, Codex, and Cursor.
The blog explains that errors in agentic chip design workflows often produce plausible outputs instead of clean failures, hiding incorrect thresholds, missing corners, or incomplete cells. Because the stages depend on one another, an early setup error can propagate into every following step. Final Liberty verification exposed missing cells and operating conditions, incorrect slew thresholds, and timing or power values outside tolerance. NVIDIA's stated conclusion is that engineering agents should be evaluated against final artifacts rather than individual tool calls.
NeMo Gym defines the task, tools, state, and verifier for each rollout so attempts start from a clean state and produce comparable results, and it records thinking traces, input and output messages, and tool actions. The optimization loop began with baseline rollouts under a versioned task and verifier in a custom environment, then the team inspected trajectories, failed tool calls, and verifier diagnostics, identified ambiguous instructions, unproductive actions, and incorrect tool selection, refined the agent's skills and MCP tool descriptions, reran the same tasks under the same conditions, and compared verified outcomes and efficiency per successful result.
The verifier assigns a composite reward from five metrics. Coverage checks whether every expected cell and PVT corner was produced. Validity checks whether the Liberty files are well-formed and usable by validation tools. Relative validation checks whether the output introduced failures beyond those already present in the reference. Accuracy is the fraction of expected timing and power values within the defined tolerance. Integrity checks whether the workflow produced the complete result rather than reusing, truncating, or reformatting existing values. The reward is the product of binary gates for coverage, validity, relative validation, and integrity, multiplied by the ratio of within-tolerance values to total output values, with zero reward if any gate fails.
The benchmark used the open source GF180MCU PDK. The dataset contains 229 standard cells across 36 PVT corners, with Solido AFS handling SPICE simulation. Each tuning run was restricted to two cells across 36 PVT corners, took about 1.5 to 2 hours, and the team ran around 25 rollouts using Claude Opus 4.8 in Claude Code as the fixed model and agent harness. Refinements included simplifying overlapping file and shell interfaces and defining tool inputs with explicit JSON schemas to make characterization tools easier to call. The post recommends evaluating final engineering artifacts rather than individual tool call success, writing a domain-specific grader or verifier that detects missing, truncated, reformatted, or reused results to prevent agent cheating, and collecting many rollouts to improve reliability through optimized skills and MCP tool interfaces. It also says the method applies to engineering agents in EDA, CAE, and other domains.
In practice, the pattern NVIDIA describes is a closed loop between a grader and prompt-level artifacts rather than model weight updates. The turn and token reductions both indicate the agent stopped wandering: clearer tool schemas and deduplicated interfaces reduce failed or repeated calls, which shortens the trajectory without changing the underlying model. A likely trade-off is that verifier quality becomes the bottleneck. Graders tuned to a specific PDK, reference dataset and tolerance can be gamed or simply fail to transfer, and integrity checks that try to catch reuse or reformatting of existing values are inherently domain-specific. The reported 55% to 95% figure is a mean composite score on representative tasks, not a production yield guarantee, so teams should expect to invest in building their own artifact-level verifiers before seeing comparable gains. One open question is how much of the improvement came from tool-description changes versus the model and harness combination, since the post fixes Claude Opus 4.8 in Claude Code throughout the tuning, which is a reasonable choice for controlling variables but limits how far the results generalize to other harnesses.
Why it matters: Chip design and EDA teams evaluating agentic automation now have a concrete, artifact-level evaluation pattern and a set of published efficiency numbers to compare against, which shifts the question from whether agents can call tools to whether their final outputs can be verified. Developers building agent harnesses in other engineering domains can reuse the structure: define a valid final artifact, write a strict verifier, inspect rollout failures, refine skills and tool descriptions, then rerun under identical conditions. The availability of NeMo Gym as an open library with a quickstart lowers the setup cost, though the reported gains are tied to a specific PDK, benchmark subset and harness configuration, so results elsewhere are an inference rather than a promise.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
AI agents offer remarkable productivity gains in complex chip design workflows, but their tendency to generate plausible yet incorrect artifacts makes...