granite-speech-5.0-2b-ONNX Speech Recognition Model: 8GB+ VRAM

granite-speech-5.0-2b-ONNX Speech Recognition Model: 8GB+ VRAM

At a Glance

Item Value
Repository ibm-granite/granite-speech-5.0-2b-ONNX
Publisher guide IBM (Granite): models and licenses
Published 2026-10-29
License apache-2.0
Formats ONNX
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

IBM has released ONNX weights for the speech language model Granite Speech 5.0 2B. This model is designed to perform automatic speech recognition (ASR) for transcribing speech input into text, and bidirectional automatic speech translation (AST) for translating speech into other languages. Using Transformers.js, in-browser inference via WebGPU is made possible.

Specifications

  • Architecture:
  • Speech encoder: Composed of 16 Conformer blocks. Trained using CTC (Connectionist Temporal Classification) with two classification heads: characters (first 256 ASCII entries for European languages and a 92 Katakana character set for Japanese) and BPE units. It also uses block attention with 4-second audio blocks and self-conditioned CTC from intermediate layers. – Speech projector: Uses a 2-layer window query transformer (Q-former). It performs a 5x temporal downsampling using 3 learnable queries per block for 1024-dimensional acoustic embeddings. Combined with the 2x downsampling by the encoder, it achieves a total 10x downsampling for a 10Hz acoustic embedding rate. – Large Language Model (LLM): Based on an intermediate checkpoint of granite-4.0-1b-base, applying YaRN scaling (scaling factor of 4, up to 16384 positions).
  • Quantization Specifications: 4-bit quantization (block size 32, asymmetric MatMulNBits / GatherBlockQuantized method) is applied.
  • Supported Languages: English, French, German, Spanish, Portuguese, and Japanese.
  • License: Apache 2.0

Performance and Quality

Regarding the quality of the ONNX version, validation results using the hf-internal-testing/librispeech_asr_dummy set have been reported. In the onnxruntime CPU EP environment, the 4-bit file recorded a 2.43% WER (2.31% when normalized). This indicates an improvement in accuracy compared to the 2.70% WER recorded by the previous 4.1 ONNX file.

Furthermore, regarding the performance of the base model granite-speech-5.0-2b, a comparison with the predecessor model granite-speech-4.1-2b is shown in the table below.

→ Scroll horizontally to see all columns

Test Set PER 4.1-2b (↓) PER 5.0-2b (↓) Cap-F1 4.1-2b (↑) Cap-F1 5.0-2b (↑)
LScln 25.70 23.20 89.71 90.16
LSoth 22.27 21.22 91.26 91.65
VoxPopuli 24.86 23.70 95.35 95.07
Earnings-22 22.87 21.64 95.19 94.98
CV-EN 9.13 9.18 96.75 96.96
CV-DE 3.66 3.65 99.50 99.51
CV-ES 11.61 11.46 95.68 95.62
CV-FR 11.00 11.43 97.25 97.20
CV-PT 7.86 7.53 98.51 98.48

From this comparison table, it can be seen that the 5.0-2b model shows improvements in PER (Punctuation Error Rate), which indicates punctuation error rates, across many test sets. Specifically, it achieves lower PER than 4.1-2b on LScln, LSoth, VoxPopuli, Earnings-22, CV-ES, and CV-PT. On the other hand, for the PER of CV-EN, CV-DE, and CV-FR, 4.1-2b shows slightly lower or equivalent figures.

Regarding Cap-F1 (Capitalization F1), which measures capitalization accuracy, 5.0-2b improves in items such as LScln, LSoth, CV-EN, CV-DE, and CV-PT, while 4.1-2b shows higher scores in VoxPopuli, Earnings-22, CV-ES, and CV-FR.

Strengths and Use Cases

This model is optimized for automatic speech recognition (ASR) and bidirectional automatic speech translation (AST) in English, French, German, Spanish, Portuguese, and Japanese. Features of the base model ibm-granite/granite-speech-5.0-2b include improved transcription accuracy in short-form English ASR and enhanced robustness against noise, reverberation, and far-field microphone audio. It also features support for input language ID prompting in multilingual ASR, improved long-form and conversational speech translation, and an expanded keyword list size (up to 2000 words) in keyword-biased ASR and AST. By using the ONNX weights, in-browser inference can be performed via Transformers.js and WebGPU.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch) — 2.3B parameters (taken from the base model ibm-granite/granite-speech-5.0-2b)

Your VRAM Quantization File size Est. memory needed
8GB (RTX 4060 / 3060 Ti, etc.) BF16 4.3GB 5.2GB

Inference engine support (architecture name matched against each project’s own model registry in its source code, checked 2026-11-17): llama.cpp: registered, vLLM: registered, MLX (mlx-lm): not registered. “Not registered" means the name is absent from that registry today, not that the model cannot run.

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

The publisher distributes this model as ONNX.

License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.

Compiled by this site’s code from the published formats and the license field. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Recent Models in the Same Size Class

Models with up to 4B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.

→ Scroll horizontally to see all columns

Model Parameters Smallest VRAM tier License Our article
ggml-org/higgs-audio-v3-stt-GGUF 2.7B 4GB apache-2.0 higgs-audio-v3-stt-GGUF Speech-to-Text Model: 4GB+ VRAM, File List (2026-11-17)
FermionResearch/Phonon-2 627M 4GB cc-by-4.0 Phonon-2 Speech Recognition Model: 4GB+ VRAM (2026-09-30)

Distributed Files

Weight files published in ibm-granite/granite-speech-5.0-2b-ONNX, listed by this site from the Hugging Face API. Sizes are the actual file sizes.

File Size
onnx/audio_encoder_q4.onnx 0MB
onnx/decoder_model_merged_q4f16.onnx 0MB
onnx/embed_tokens_q4f16.onnx 0MB

How to Get It

Available from the Hugging Face repository ibm-granite/granite-speech-5.0-2b-ONNX. It supports in-browser inference using Transformers.js (version 4.3 or later), and the official granite-speech-webgpu space is published as a demo. Specific usage code examples with Transformers.js are posted on the model card.

Related Articles

What to Read Next

Sources