Announcing the Special Interest Group (SIG) for Ray on Accelerators

The PyTorch project announced a new Special Interest Group (SIG) focused on running Ray workloads on hardware accelerators. The group is meant to be a public, vendor-neutral forum that brings together hardware vendors, Ray contributors and application developers to set a shared roadmap for accelerator support. The announcement frames the move as building on work the community has already done: Ray now supports more than a dozen kinds of accelerators, spanning NVIDIA and AMD GPUs as well as TPUs, Trainium, Gaudi and others.
The stated motivation is that the hardware landscape has fragmented. When Ray began at UC Berkeley more than 10 years ago, distributed computing largely meant CPU clusters running frameworks like Apache Hadoop and Apache Spark, and GPUs were just starting to move from graphics and scientific computing into machine learning. The announcement points to a spread of current devices with quite different strengths: NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs for mixed training, inference, graphics and video work; NVIDIA HGX B200 for larger memory capacity; NVIDIA GB200 NVL72, which links 72 Blackwell GPUs for communication-heavy training and inference; AMD MI355X for large-model memory capacity; Google Ironwood TPUs for very large workloads; and NVIDIA Groq 3 LPUs aimed at low-latency inference.
The initial roadmap covers three areas. The first is development and testing infrastructure. The group wants a documented process for bringing cloud- and vendor-hosted accelerators into Ray's testing setup, so contributors can reproduce and debug failures on the devices their changes affect. It also plans maintained environments with explicit driver, runtime and dependency versions, broader CI coverage spanning pull-request checks plus nightly and release testing, and defined ownership for keeping environments and integrations healthy across releases. The announcement singles out distributed workloads, failure recovery and performance regressions as things that coverage should include, not just basic device functionality. It also notes that as agentic software engineering speeds up development, reliable feedback from real hardware becomes more important for both humans and coding agents.
The second area is hardware-aware scheduling. The argument here is that an accelerator node is no longer just a count of identical devices; performance depends on how devices are interconnected, how they relate to CPUs and NICs, and how nodes group into racks or TPU slices. Planned work includes vendor-neutral interfaces for labeling hardware, describing architecture and specs, and identifying SKUs consistently, including new ones at launch, so applications can target capabilities rather than write vendor-specific code. The group also wants a documented extension point and checklist for vendors adding an accelerator. Discovery should capture topology and architecture, not just device type and count, with clear ownership per integration. Because some details such as rack or slice membership come only from the platform, cloud integration is part of the plan, as is visibility into devices and their connections on the Ray dashboard.
The third area is networking and I/O transport. Ray's object store was originally built for CPU data, so accelerator tensors had to be copied to host memory, serialized and sent over standard host networking to reach another device. Ray Direct Transport (RDT) was introduced to avoid that: actors pass references to tensors through Ray while the data moves directly between devices over high-performance transports such as UCX and RDMA. The group wants broader RDT support across accelerators, frameworks and interconnects so Ray can pick the fastest path, whether intra-slice TPU ICI, scale-up NVIDIA NVLink or scale-out GPUDirect RDMA. It also targets clean initialization, teardown and error recovery for each transport, plus validated deployment recipes, benchmarks and transport metrics.
The division of responsibility is explicit. Core scheduling and placement logic stays in Ray Core, while the SIG defines plug-in interfaces, CI testing and maturity tiers for accelerators. Vendors and cloud providers contribute and maintain their own device plugins, transports and day-zero support for new SKUs, and framework authors and practitioners are expected to ground the roadmap in real workload needs. Participation channels named in the announcement are the Ray Slack, community meetings and accelerator-related issues and pull requests on GitHub.
In practice, the value of a group like this depends on whether vendors actually staff the CI capacity and plugin maintenance it calls for, since a vendor-neutral interface only helps if every supported device gets tested regularly. A likely trade-off is pace versus breadth: defining common abstractions for hardware that differs this much takes time, and early interfaces may need revision as new accelerator types appear. Treating topology as a first-class scheduling input is a meaningful change, since it moves Ray's placement decisions beyond simple device counts toward locality-aware choices, which is useful for communication-bound training but adds complexity to discovery and to the data cloud platforms must supply.
Why it matters: teams running training, inference or reinforcement learning on Ray across mixed hardware should expect more emphasis on reproducible CI, topology-aware placement and faster device-to-device transfer through RDT, which affects how portable a workload is across vendors. For hardware and cloud providers, the SIG is a defined channel to get new SKUs supported at launch rather than after the fact, though it also implies ongoing maintenance obligations. Whether this yields consistent day-zero support is inference, not something the announcement promises beyond the stated roadmap.
Category: Developer tools
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
Today, we are announcing the Special Interest Group (SIG) for Ray on hardware accelerators. When Ray was created in UC Berkeley over 10 years ago, the computing landscape looked very... The post Announcing the Special Interest Group (SIG) for Ray on Accelerators appeared first on PyTorch .