Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint

IBM has released Granite Switch 4.1, an update to its Granite Switch project, which it says was first introduced last spring. The new models, Granite-Switch-4.1-8B and Granite-Switch-4.1-30B, are distributed as "batteries included" Switch models: a frozen Granite base model compiled together with a predefined set of adapters so that a single hosted endpoint can handle natural-language chat, general and specialized decisions, and speech-to-text transcription.

Three technologies are composed into each released model. The first is a new Decision aLoRA aimed at general-purpose decision-making. The second is a specialized Guardian aLoRA that IBM says outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions and Clef-Flash on average on safety-related decision tasks. The third is Granite-Speech-5.0-Turbo-CTC, described as the fastest ASR model on the OpenASR Leaderboard, embedded for transcription work. All three, plus the underlying base model, occupy approximately the same amount of VRAM on a GPU as the base model would on its own — IBM frames this as more capabilities with fewer GPUs.

The mechanism behind that efficiency is the adapter format. The Switch adapters are built as cache-efficient Activated LoRAs, or aLoRAs, which means the KV cache produced by a base-model generation can be reused for any subsequent adapter-based decision. In transformer inference, the KV cache stores the key and value tensors for tokens already processed so they do not have to be recomputed on each step; sharing it across adapters avoids rebuilding that state when the model switches tasks. IBM also cites its team's recent vLLM contributions for speeding up low-rank adapter inferencing, and a co-located ASR encoder, as reasons the models can move quickly between a chat request, a decision request and a spoken request.

As a concrete illustration, IBM says Granite-Switch-4.1-8B can transcribe audio, generate summaries, screen for harmful content and test more than 15 generic flags, all in real time from a single endpoint.

Decision models are not new to IBM. An earlier Granite Libraries release provided a repository of specialized decision-model adapters and other task adapters that can be composed into custom Granite Switch models. IBM notes that while Jev and other closed decision models emphasize generality, many of its specialized granitelib adapters can beat generalists on specialized classification tasks. Because adapters share the base model's KV cache, adding specialists alongside a generalist decision adapter costs almost nothing, so a deployment can get broad coverage from the generalist and higher accuracy from specialists on the tasks that matter most.

IBM also reports a head-to-head comparison it calls a race, between Granite-Switch-4.1-30B (a frozen base Granite plus four specialized aLoRAs) and Jev-27B (a frozen base Qwen model plus one generic decision LoRA). In that test, IBM says Granite-Switch-4.1-30B completed a real-world, five-step RAG workflow much faster than the competitor. The company does not publish latency figures for that workflow in this announcement.

Composition is not restricted to IBM's prebuilt bundles. The 4.1-8B and 4.1-30B releases ship with a default adapter set, but IBM says anyone can compose their own Granite Switch model, mixing and matching adapters from Granite Libraries. It plans an Adapter SDK for training adapters in Mellea, alongside tutorials and Granite Switch support. Suggested uses include specialized classification adapters to augment decision workflows or adapters for specialized language tasks. As an extreme example, IBM says a custom adapter can teach a 3B-parameter version of Granite to play the classic game Doom.

That Doom experiment doubles as a performance anecdote. IBM says its custom 3B Granite Switch model made split-second navigation decisions at 60 to 100 decisions per second, compared with Jev's reported 10 decisions per second, citing a typesafe.ai post on system-one models as the source for the Jev figure. Unlike Jev, IBM adds, the Granite Switch model can also narrate and answer questions while playing.

Resources listed for getting started include the Granite Switch 4.1 model cards, a speech-to-decisions demo, the Granite Switch GitHub repository, the original Granite Switch and Granite Libraries release blog, and a Colab notebook for running your own aLoRA-versus-LoRA race.

For developers, the practical claim to scrutinize is the VRAM profile: if a bundle of three adapters really fits in roughly the footprint of the base model, then teams that already serve a Granite model can add decision and transcription capabilities without adding GPU capacity, which is the main cost driver in self-hosted inference. A natural trade-off is that prebuilt bundles bake in IBM's adapter choices, so teams with specialized needs will still want to compose their own stack once the Adapter SDK and Mellea tooling arrive. The headline benchmark comparisons — the OpenASR ranking, the Guardian results against named closed models and the RAG race against Jev-27B — come from IBM's own reporting here, and, as with most vendor benchmarks, independent reproduction on your own workloads is the only way to know how the numbers translate.

Why it matters: teams serving Granite models on a single endpoint could gain decision and speech capabilities at little extra memory cost, which matters most for cost-constrained self-hosted deployments. The planned Adapter SDK and Mellea training path suggest IBM wants third parties to build the specialized adapters rather than supply them all itself, a bet that may shift work toward adapter training and away from full fine-tunes. Whether the speed and quality claims hold outside IBM's benchmarks is the open question worth testing before committing production capacity.