AivexaNewsSearch
AI news for builders and product teamsChecked every hour

A Closer Look at Phonon TTS. Launching High-Quality On-Device Speech That Tops the Benchmarks.

Collected Nov 5, 2026

Gradium released Phonon, a small text-to-speech model that runs entirely on the user's device without a server or network connection, and has now published a detailed account of the launch. The initial release came two weeks ago, following an April announcement and roughly six months of work with partners and early users. The company reports more than 200,000 downloads in the first two weeks and says the model is still trending on Hugging Face.

Phonon ships in two sizes, roughly 65 MB and 75 MB, together covering English, French, German, Spanish and Portuguese. There are more than 100 built-in voices, and users can create their own through Gradium Studio. The team also highlights handling of phone numbers, dates and codes, inputs that commonly trip up TTS systems.

The licensing splits by model. Phonon 31m speaks English and is Apache 2.0, so it can be used free of charge, including in commercial products. The other models are CC BY-NC, free for non-commercial work; commercial use of those is priced by number of users rather than by volume of speech generated. That structure makes the small English model the obvious entry point for production work, while multilingual support sits behind a non-commercial ceiling unless a commercial arrangement is made.

Architecturally, Phonon descends from the Pocket TTS lineage and uses flow matching to generate audio, a different approach from the modeling techniques common in other TTS systems. Gradium credits this choice for reaching its quality at such a small size, and notes it brought training difficulties: the training was initially numerically unstable and needed several tricks before producing quality results across all five languages.

On the Hugging Face Open TTS Leaderboard, a new project that evaluates models across 23 languages using intelligibility (word error rate of outputs), speed (inverse real-time factor and time-to-first audio) and naturalness (UTMOS for English, KOMOS for Korean), Phonon ranks first in German and Spanish on intelligibility and sits in the top three in every language evaluated — placed alongside models up to 100 times larger. On speed, Gradium says the models top the CPU streaming benchmark for time-to-first audio. The leaderboard covers both fixed-voice and voice cloning, the latter adding a speaker similarity metric, and includes a Listen tab for human comparison and voting.

For inference, Gradium built a dedicated framework called ptts, aimed at strong performance across a wide range of hardware. It supports more than ten backends spanning CPU (Intel, Arm, WebAssembly), GPU (CUDA, Vulkan, Metal, WebGPU) and NPU (Apple Core ML, Snapdragon). The model also runs on phones, laptops, single-board computers and in a browser tab.

Phonon is additionally available in Hugging Face Transformers from v5.21 as its reference PyTorch implementation. The division of labor is clear: ptts targets single-stream on-device use, while Transformers targets GPU throughput with batched generation and full-graph torch.compile, suited to synthesizing speech datasets at scale. Because Pocket TTS was added to Transformers first, Phonon sits on top as a modular model of about 200 lines describing only what differs.

In practice, Phonon generates audio latents autoregressively: each step's output latent feeds back as the next input, and the training loss is computed on those latents. For a developer, the useful analogy is that latents play the role language-model tokens do, so the codec mapping audio to latents acts as a tokenizer. Transformers ships it that way, with an audio tokenizer held by the processor beside the text tokenizer. The reference checkpoint in the example, gradium/phonon-31m-en, carries 30,788,354 model parameters and a 10,305,313-parameter audio tokenizer.

A voice is not stored as a recording but as a tensor of speaker latents. Phonon never encodes audio directly with its audio tokenizer, which ships decoder-only, and voices come precomputed from the gradium/phonon-voices repository. A voice file loads as an array shaped by frames and 512 channels. The prompt layout — voice positions, then text, then a token opening the audio — ships with the checkpoint as a chat template, following the pattern of other multimodal models in Transformers. You write a standard conversation with an empty audio item where the voice belongs, pass the voice latents through processor_kwargs, and the processor expands the placeholder to one position per frame, normalizes and tokenizes the text, and batches the inputs. Generation returns latents that the processor decodes to audio and can save directly as a wav file. The upshot is that the same conversational code works across checkpoints, with voice swapped by data rather than code.

The launch shipped with demos spanning several contexts: an Apple Watch with Wi-Fi disabled walking through a coffee brew, an iPhone with no signal counting reps, a robot called Lamp from Autonomous speaking in character, a browser-based offline dino runner turned radio show, a sci-fi game voiced on the player's machine, and Proust read aloud page by page. The Hugging Face team built the Transformers integration so the model was usable there from day one. Third parties have since added an ExecuTorch conversion that runs Phonon on a visitor's GPU in the browser, Micdrop for Node.js apps, and Spaces such as smallest-s2s.

Looking ahead, Gradium says Phonon can speak with any voice: clone one from a short recording with the speaker's consent, or describe a new voice in plain text in Gradium Studio, then download the file and load it for fully on-device use. The company also trains custom models with design partners, offering smaller variants for tight memory budgets, additional languages such as Korean, Arabic or Japanese, and fine-tuning on a product's own vocabulary. Phonon is Gradium's first open release.

Why it matters: Developers building voice features have usually accepted a cloud round trip and the privacy and latency costs that come with it. A 31M-parameter model under Apache 2.0 that ranks first or near-first on intelligibility and time-to-first-audio, per the leaderboard results reported here, makes on-device speech a realistic default for English-language products, and the Transformers integration lowers the cost of trying it. Teams needing multilingual or cloned voices will have to weigh the non-commercial licensing or negotiate commercial terms — a likely friction point, in my reading — while the tiny footprint and sub-second streaming suggest the main competitive pressure falls on hosted TTS APIs.

Read at Hugging Face Blog

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt