AivexaNewsSearch
AI news for builders and product teamsChecked every hour

TornadoVM 7 Enables GPU-Native Workloads from Java Applications

Collected Oct 21, 2026

TornadoVM, an open-source framework that lets Java programs run accelerated code on multi-core CPUs and GPUs, has shipped version 7.3, described by InfoQ as a major milestone for the project. The write-up is by Ben Evans; InfoQ had previously covered TornadoVM in December 2025 and in 2022. TornadoVM does not replace the JVM. It runs alongside any mainstream JDK distribution, including OpenJDK, GraalVM, Temurin, Red Hat OpenJDK, Oracle JDK and Zulu, and offloads selected Java code to hardware backends while managing memory between the Java heap and the accelerators.

The headline architectural change in version 7 is the switch of the backends from the Java Native Interface to the Foreign Function & Memory (FFM) API. FFM, finalised in recent JDKs, is the modern supported route for calling native code and working with off-heap memory from Java, and it is intended to be safer and more maintainable than JNI. Alongside it, the experimental JVM Compiler Interface dependency has been removed. According to the team, TornadoVM now obtains all compiler metadata through a reflection, ASM and Unsafe layer, which serves as a single uniform path on every supported JDK: 21, 25, 26 and 27. This suggests the project is deliberately trading some reliance on internal compiler hooks for a more portable base, which should make it easier to track new JDK releases; a likely trade-off is that reflection and Unsafe carry their own fragility and may draw warnings under stricter future JDK policies.

Development and deployment are meant to be decoupled. During development TornadoVM compiles Java source to OpenCL and Apple Metal backends, so a developer can work on a laptop using its local hardware, then deploy to a server with Nvidia GPUs for larger gains. The team frames this as a Write Once, Run Anywhere idea applied to modern machine learning workloads. The reasoning is straightforward: OpenCL and Metal cover many client machines, particularly Apple hardware, while Nvidia hardware dominates many server and cloud environments, so a single Java codebase can target both.

On the Nvidia side, the release deepens integration with an optimized CUDA backend. It adds support for the cuTile programming model through a new TileContext, and allows CUDA libraries such as cuBLAS and CUTLASS to be invoked directly from inside TornadoVM TaskGraphs. Developers can therefore substitute JIT-generated kernels with CUDA library calls, or chain the two, depending on whether performance or portability matters more for a given stage. The same capability is extended to Apple's MXL library through what the project calls its hybrid API. Multi-GPU execution is also now supported.

Not every Java computation gains from offloading. The strongest candidates are data-parallel workloads, such as loops whose iterations have no dependencies on each other, because those iterations can run concurrently on the accelerator. A recent goal of the project is to give Java developers idiomatic access to libraries Nvidia has tuned per hardware platform, spanning linear algebra, Fast Fourier Transforms for signal analysis, deep neural networks and hand-optimized sparse matrix multiplication.

TornadoVM plugs into existing Java and data infrastructure. It has library integrations with Quarkus, Lang4j, Flink, Spring AI and others, and is distributed through SDKMan. The Flink case shows the intended pattern: a developer writes a normal query with the Table API and sets a configuration flag indicating that the query may be accelerated on a GPU. When the computation reaches the task manager, the framework checks whether a GPU is present on that node; if so, a library provider analyses the query's intermediate representation to decide whether the computation actually suits GPU execution. The interesting detail is that the decision is made at runtime per node, which fits heterogeneous clusters but means the same query can take different execution paths depending on where it lands.

Java programmers have four routes to Nvidia GPUs in TornadoVM. The simplest is the parallel API, an annotation on a loop that hints to the compiler that iterations may be parallelised. That alone proved insufficient over time for production-grade performance, so the kernel context API and the new TileContext were added. In practice this reflects a familiar pattern in Java acceleration work: a low-effort annotation gets adoption, while more explicit control is needed when the last increment of throughput matters.

Released alongside 7.3 is jitLLM v2.0, a next-generation Java AI inference engine built on TornadoVM. The team positions it as a direct performance competitor to llama.cpp and Ollama, and it ships as a single JAR file. Both projects are being commercialized through Beestream.ai, a spin-off founded by the TornadoVM team. Further details are available on the TornadoVM site and in recent Devoxx Belgium talks, including one titled on tapping into the Nvidia ecosystem from Java with TornadoVM's hybrid API.

Why it matters: Java teams that already run on the JVM can now reach GPU and CUDA library performance without rewriting services in C++ or Python, and the FFM-based backend plus a single metadata path across JDKs 21 through 27 reduces friction when upgrading runtimes. Data platform teams using Flink or Spring AI have a plausible route to per-node GPU acceleration, though the runtime suitability check means gains are not guaranteed for every query. Inference-focused teams may want to benchmark jitLLM against established tools themselves, since the competitive claims come from the project's own team, and the commercial spin-off suggests some capabilities may increasingly be shaped by commercial priorities.

Read at InfoQ · AI, ML & Data Engineering

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

TornadoVM, an open-source heterogeneous programming framework for Java, has reached version 7.0. It automatically accelerates Java programs on multi-core CPUs and GPUs and provides a rich GPU programming ecosystem in Java that can be used with any mainstream JDK distribution e.g. Temurin, Red Hat OpenJDK, Coretto, Oracle, GraalVM or Zulu. By Ben Evans