AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Runway's open-source Praxis-1 turns video AI into a robot controller

Collected Nov 14, 2026

Runway publicly released Praxis-1, its AI model for controlling robots, with open weights and source code under the Apache 2.0 license, which permits commercial use. The company had first introduced the model in late September but restricted access to select partners. The release now comes with a detailed research report containing new performance tests. Weights are downloadable from Hugging Face after signing up and accepting access terms, and the code is on GitHub.

Praxis-1 takes in camera images, text instructions and information about a robot's position. From these inputs it predicts how the environment will change and calculates the movements that match, adjusting control continuously as new camera frames arrive. Runway describes it as a World Action Model: rather than mapping instructions straight to joint commands, it models how the scene will evolve and derives actions from that prediction. This is the same broad idea behind several recent robotics efforts that pair a learned forward model of the world with an action head, and it contrasts with more conventional imitation-learning policies that learn a direct input-to-output mapping.

What distinguishes the approach is where the training signal comes from. Praxis-1 builds on a video model that learned how objects move and interact from ordinary footage, which Runway then fine-tuned with videos of human actions and robot data. The pitch is that this could reduce reliance on expensive robot-specific datasets, which have traditionally been collected purely to train such systems. Since September, Standard Bots, Noble Machines and Ultra have been testing the model on a humanoid robot, an industrial robot arm and a dual-arm warehouse robot respectively.

Some specialised robot data is still needed. To adapt Praxis-1 to a Franka robot arm, Runway used about 277 hours of recordings from the public DROID research dataset, which shows robots grasping and moving objects. For a dual-arm robot it used between 1,400 and 2,600 hours of training material. DROID and data from other robot systems had already featured in earlier training stages. Whether the approach genuinely needs less robot data overall than comparable models without general video pretraining remains an open question, and that is the claim most worth watching in independent replications.

On Nvidia's RoboLab-120 leaderboard, which tests how reliably models sort, grasp or stack objects across 120 simulated tasks, Praxis-1 posts a 46.2 percent success rate. FLUX 3 Action, another World Action Model, scores 42.9 percent under the same conditions, putting Praxis-1 top of that category. Across the full leaderboard it sits fifth, behind two AI agents and two robot models built on different architectures. Runway conducted all the tests itself and the results have not been independently verified.

Runway also ran real-hardware tests on a Franka arm after fine-tuning on DROID data. Both models attempted the same ten tasks, such as placing a banana on a plate or closing a drawer, with ten tries each. Praxis-1 succeeded 88 times against 80 for FLUX 3 Action.

A separate experiment probed transfer to an unfamiliar robot: a mobile platform with a gripper arm that Google had previously used to develop RT-1, with recordings from it absent from Praxis-1's earlier training. After fine-tuning on roughly 95 hours of robot data, the model reached an average 80 percent success rate in simulated tests spanning four task types, including grabbing soda cans, opening and closing drawers and placing an apple in a drawer. Google's original RT-1 scored 52.4 percent on the same benchmark. This suggests that video pretraining gives the model a useful head start when it meets a new embodiment, though the comparison is against an older baseline rather than a current competitor.

On hardware, the standard model requires a GPU with 80 GB of memory. A compressed version should fit on an RTX 5090 with 32 GB of VRAM according to Runway's memory estimates, although the GitHub documentation notes actual peak memory usage has not been measured. Pre-adapted versions for different robots are available alongside the base model; the base itself needs additional training before it works with a new robot system.

The limitations are explicit in Runway's report. Praxis-1 occasionally confuses similar-looking objects and struggles with vague instructions. For longer tasks, individual steps must be spelled out, because the model cannot yet reliably plan complex action sequences on its own. Runway positions Praxis-1 primarily as a system for fast execution of robot movements, and envisages it working alongside general-purpose AI models that decompose complex tasks into individual steps. In practice, that means a plausible division of labour: a language or planning model decides what to do, and Praxis-1 handles the moment-to-moment control. A likely trade-off is that this architecture keeps the control model small and fast but pushes integration work, prompt or plan formatting, and error recovery onto whoever assembles the full stack.

Why it matters: An Apache 2.0 robotics foundation model with downloadable weights lowers the barrier for teams that want to experiment with manipulation without training from scratch, and the 32 GB estimate puts fine-tuning within reach of a single high-end workstation rather than a cluster. Developers should treat the benchmark numbers as vendor-reported until independent runs appear, and should expect to supply their own robot data and a separate planner for multi-step tasks. Companies building warehouse or industrial automation are the clearest near-term audience, since the cited partner tests already cover an industrial arm and a dual-arm warehouse robot.

Read at The Decoder

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

Runway releases its robotics model Praxis-1 with open weights and source code under the Apache 2.0 license. The model uses knowledge from video training for robot control and leads the World Action Model category on Nvidia's RoboLab-120 benchmark. It can't yet plan complex action sequences on its own, though. The article Runway's open-source Praxis-1 turns video AI into a robot controller appeared first on The Decoder .