Phonon: A Lightweight, Fully On-Device Text-to-Speech Model

At a Glance
| Item | Value |
|---|---|
| Publisher | Hugging Face Blog |
| Published | 2026-11-05 |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
Technical details have been released for “Phonon," a small-scale text-to-speech (TTS) model that operates entirely on-device. Phonon is a text-to-speech model that generates audio directly from text input by the user, without requiring any servers or external network communication.
Despite its extremely small model size of approximately 65MB to 75MB, it achieves high quality and can be executed locally in diverse environments, including smartphones, PCs, single-board computers, and even within browser tabs.
Specifications
The main specifications and conditions for Phonon are as follows:
- Parameter count and model size:
- Phonon 31m: 30,788,354 parameters for the model, 10,305,313 parameters for the audio tokenizer (size is approximately 65MB). – Phonon 40m: Parameter count not disclosed (size is approximately 75MB).
- Architecture: Belongs to the Pocket TTS lineage and employs Flow matching technology for audio generation.
- Output specifications: Audio waveforms (generates audio latent representations autoregressively and reconstructs them via a decoder). Supports reading of special symbols and notations such as phone numbers, dates, and codes.
- Supported languages: A total of 5 languages including English, French, German, Spanish, and Portuguese (Phonon 31m is English only; Phonon 40m supports all 5 languages).
- Voice features: Includes over 100 built-in voices. Custom voice creation is possible through voice cloning from short recordings or voice design features using text descriptions.
- License conditions:
- Phonon 31m: Apache 2.0 license (free and free to use/embed, including commercial use). – Phonon 40m: CC BY-NC license (free for non-commercial projects. For use in commercial products, a licensing scheme based on the number of users rather than generation volume is applied).
Performance
According to the publisher, evaluation results for Phonon on the “Open TTS Leaderboard" provided by Hugging Face have been presented. This leaderboard evaluates open TTS models across 23 languages based on intelligibility (Word Error Rate), speed (inverse real-time factor and time-to-first audio), and naturalness. Word Error Rate (WER) is a metric that measures the error rate per word in transcriptions, where lower values indicate better performance.
In terms of intelligibility, Phonon ranked first in both German and Spanish. Furthermore, in all languages evaluated, it ranked within the top 3, alongside large models with up to 100 times its parameter size.
It also performs exceptionally well in speed evaluations, recording the top position in the “time-to-first audio" category for CPU streaming benchmarks. While it achieves top-tier intelligibility and response speed despite its tiny model size, it is assumed that there are trade-offs regarding subtle nuances and expressive power in specific domains when compared to massive models.
Strengths and Use Cases
Phonon is suited for real-time speech synthesis and reading applications in fully local environments without any network communication, as well as various on-device applications.
The official blog highlights the following specific use cases:
- Voice assistants and guidance in offline environments: Examples include reading step-by-step coffee brewing instructions on an Apple Watch with Wi-Fi turned off, or counting workout reps on an iPhone in areas without service.
- Interactive dialogue, robotics, and gaming: Includes character voices like a sorting hat for the autonomous robot “Lamp," audio output for indie games running on local PCs, and radio-style commentary for offline browser games.
- Long-form reading and document narration: Use cases such as reading long texts, like reading Marcel Proust’s works one page at a time, are mentioned.
- Accurate pronunciation of special notations: Supports accurate reading of phone numbers, dates, and various code notations that many TTS systems often misread.
- Creation and application of custom voices: In addition to over 100 built-in voices, users can utilize voice cloning from short recordings (with speaker consent) or the voice design feature in Gradium Studio, which generates voices based on text descriptions. By loading the created voice file (speaker latent representation tensor) locally, any voice can be synthesized on the device.
How to Get It
Phonon can be obtained and used via multiple libraries and inference engines depending on the use case and execution environment.
ptss has been developed as a dedicated lightweight inference framework. ptss is optimized for single-stream execution on devices and supports over 10 backends, including CPU (Intel, Arm, WebAssembly), GPU (CUDA, Vulkan, Metal, WebGPU), and NPU (Apple Core ML, Snapdragon®).
It has also been integrated into Hugging Face’s Transformers (v5.21 and later) as an official PyTorch reference implementation. Transformers supports batch generation using GPUs and graph optimization via torch.compile, making it suitable for synthesizing large audio datasets.
An example of Python code to load the model and processor using Transformers is as follows:
from transformers import AutoProcessor, GradiumPhononTTSForConditionalGeneration
model_id = "gradium/phonon-31m-en"
processor = AutoProcessor.from_pretrained(model_id, device_map="auto")
model = GradiumPhononTTSForConditionalGeneration.from_pretrained(model_id, device_map="auto")
Voice data used for speech is downloaded and utilized as precomputed data in safetensors format from the gradium/phonon-voices repository. You can pass the voice data to the apply_chat_template method of AutoProcessor, and save the audio as a file (.wav) by decoding the latent representation generated by the generate method.
Additionally, as part of the ecosystem, projects such as ExecuTorch conversion for GPU inference in browsers, Micdrop for Node.js applications, and smallest-s2s on Hugging Face Spaces are underway.
Related Articles
- Mistral Large 4 Released: Multimodal MoE Model
- Run GGUF Models Directly in Transformers with llama.cpp Support
- Black Forest Labs Releases FLUX.3 Image Klein
- BotPsy: 0.8B On-Device Agentic Model by Tether AI Research
What to Read Next
- How to read WER → Benchmark glossary

