Using AI Agents to Teach Robots Like Microduck to Walk in Simulation

NVIDIA published a developer walkthrough of an end-to-end workflow for teaching a small four-legged robot called Microduck to cross a simulated obstacle course. The pipeline spans AI-assisted planning, 3D asset generation, physics-based environment assembly, reinforcement learning and local evaluation, with NVIDIA RTX Spark serving as the local development hub and cloud-scale training done in NVIDIA Isaac Lab. Two agents, Hermes and NVIDIA Nemotron 3.5 Bolt, turn the initial prompt into a structured project plan, and a third, Astra, builds the simulation assets and assembles the scene. The post also credits a long list of NVIDIA collaborators across the teams involved.
The workflow starts with a text prompt describing the course and what the robot must learn. On RTX Spark, Hermes and Nemotron 3.5 Bolt convert it into a staged plan covering research, concept generation, asset identification, handoff, asset creation, and assembly and testing. The plan also calls for a short review of Microduck's repository and documentation to establish its capabilities, available resources and open questions, and it stretches beyond asset creation to environment construction, robot preparation, simulation, training, validation and deployment. A local image-generation model produces a photorealistic concept of the course, which the agents then analyze into an inventory of reusable objects. Each item gets an identifier, a description of visible features and a cropped reference image, and the inventory records where objects are partly cut off or hidden so later steps can tell observed detail from assumption. Hermes then packages the plan, research links, concept image, inventory and crops into a handoff and delegates to Astra.
Astra turns those references into 3D assets. Because crops from a larger concept image are small and low-resolution, it first uses cloud image editing to isolate objects and produce clearer, higher-resolution images, then checks the enhanced versions against the originals for consistent silhouettes, proportions, colors and distinctive features. Dimensions and hidden surfaces that the images cannot settle are logged as modeling assumptions. Base geometry and an editable CAD structure are built programmatically in FreeCAD, fine detail and materials are added in Blender, and the results are saved as editable models and exported as USD assets with previews and output checks. Teams that would rather not generate everything from scratch can pull from SimReady Central Library, which the post describes as a free, ungated collection of more than 2,000 simulation-ready OpenUSD assets.
The assembled course lives in NVIDIA Isaac Sim, which supplies rendering and sensor simulation through Omniverse libraries, while Newton models the robot's physical behavior, including movement and balance. In simulation the robot is described through links, joints and actuators: links are connected bodies, joints constrain how they move relative to each other, and actuators drive motion, together defining how policy actions map onto the simulated machine. Simulated sensors, including a head camera, report on obstacles ahead, and the simulation records the consequences of each action, whether posture is held, progress is made, or the robot stumbles.
Training runs in Isaac Lab, an open-source framework that executes many copies of the environment in parallel so a single shared policy can be improved from all of their experience at once. For a given task, the developer defines observations, available actions and rewards, and implements the task with Isaac Lab APIs; Newton handles the resulting motion and contact with ground and obstacles. The loop is simple to state: observations feed the policy, the policy emits actions, the simulation advances the robot, and new observations come back. Early attempts produce unstable steps, collisions and falls, and the reward signal pushes the policy toward more successful behavior.
The reported experiment focuses on ramps and uneven surfaces: the robot must cross a curved ramp while keeping balance. Starting from a locomotion policy pretrained in the stock Isaac Lab Microduck example environment, additional training adapts it to ramps and obstacles using head-camera depth and robot-state observations. That adaptation run used 4,096 parallel environments on a single NVIDIA RTX PRO 6000 Blackwell Workstation Edition GPU, a concrete data point on how much simulation throughput one high-end workstation card can sustain.
After training, the policy comes back to RTX Spark for evaluation in Isaac Sim, running locally and stepping through the scene as observations arrive. The point of this stage is to separate learning from assessment: during training the policy keeps changing, whereas evaluation looks at one fixed policy to see whether it detects obstacles, adjusts its gait and finishes the course. Teams are advised to log completions, falls, collisions and the spots where progress stalls, and to pair those numbers with rollout videos to decide whether to revise the environment or the training setup. Installing Isaac Sim on RTX Spark uses Windows x86 binaries that run through Windows x86 emulation on Arm.
The post stops short of detailing hardware deployment, but describes the groundwork for sim-to-real transfer. Isaac Lab supports Better Actuator Models, which represent servo behavior such as command delays, mechanical backlash in the drivetrain, and voltage drops as motors draw from a shared battery; matching those settings to measurements taken on the real robot can narrow the gap. Domain randomization then varies physical parameters and observation noise within realistic ranges so the policy practices handling conditions it did not see in training, and hardware tests point to where the next round of simulation and training should focus. A final figure shows a locomotion policy trained in Isaac Lab driving the physical Microduck.
This suggests the most interesting structural claim is the division of labor: generative agents handle the tedious front end of planning and asset construction, GPU-parallel simulation handles policy learning, and a local workstation handles inspection. Each stage produces reviewable artifacts, such as prompts and provenance records, cropped references, assumption logs and rollout videos, which matters because agent-generated assets are exactly the kind of thing that silently drifts from the spec. A likely trade-off is that this convenience depends on an RTX-class local machine plus cloud or lab GPU capacity, and the post itself notes that simulation-to-reality differences still need refinement beyond what is covered.
Why it matters: Robotics teams evaluating reinforcement learning now have a concrete reference for how far an agent-assisted pipeline can carry a project, from a prompt to a trained ramp-walking policy, without hand-building every asset. The 4,096-environment run on a single RTX PRO 6000 Blackwell card is a useful benchmark for budgeting training hardware, and the emphasis on measured actuator models and domain randomization signals that the hard work has moved to calibrating simulation against the physical robot. Developers outside robotics can read it as a pattern for orchestrating specialized agents around a structured handoff.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
Teaching a robot to navigate an obstacle course requires more than a walking policy. It must perceive its surroundings, maintain balance, and adjust its actions...