{"id":14028,"date":"2026-11-17T11:02:52","date_gmt":"2026-11-17T02:02:52","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/?p=14028"},"modified":"2026-11-17T11:12:10","modified_gmt":"2026-11-17T02:12:10","slug":"ibm-granite-speech-5-0-2b-onnx","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/ibm-granite-speech-5-0-2b-onnx\/","title":{"rendered":"granite-speech-5.0-2b-ONNX Speech Recognition Model: 8GB+ VRAM"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/huggingface.co\/ibm-granite\/granite-speech-5.0-2b-ONNX\">ibm-granite\/granite-speech-5.0-2b-ONNX<\/a><\/td>\n<\/tr>\n<tr>\n<td>Publisher guide<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/publisher-ibm-en\/\">IBM (Granite): models and licenses<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-10-29<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>apache-2.0<\/td>\n<\/tr>\n<tr>\n<td>Formats<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-onnx-en\/\">ONNX<\/a><\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>IBM has released ONNX weights for the speech language model <code>Granite Speech 5.0 2B<\/code>. This model is designed to perform automatic speech recognition (ASR) for transcribing speech input into text, and bidirectional automatic speech translation (AST) for translating speech into other languages. Using Transformers.js, in-browser inference via WebGPU is made possible.<\/p>\n<h2>Specifications<\/h2>\n<ul>\n<li>Architecture:<\/li>\n<li>Speech encoder: Composed of 16 Conformer blocks. Trained using CTC (Connectionist Temporal Classification) with two classification heads: characters (first 256 ASCII entries for European languages and a 92 Katakana character set for Japanese) and BPE units. It also uses block attention with 4-second audio blocks and self-conditioned CTC from intermediate layers. &#8211; Speech projector: Uses a 2-layer window query transformer (Q-former). It performs a 5x temporal downsampling using 3 learnable queries per block for 1024-dimensional acoustic embeddings. Combined with the 2x downsampling by the encoder, it achieves a total 10x downsampling for a 10Hz acoustic embedding rate. &#8211; Large Language Model (LLM): Based on an intermediate checkpoint of <code>granite-4.0-1b-base<\/code>, applying YaRN scaling (scaling factor of 4, up to 16384 positions).<\/li>\n<li>Quantization Specifications: 4-bit quantization (block size 32, asymmetric <code>MatMulNBits<\/code> \/ <code>GatherBlockQuantized<\/code> method) is applied.<\/li>\n<li>Supported Languages: English, French, German, Spanish, Portuguese, and Japanese.<\/li>\n<li>License: Apache 2.0<\/li>\n<\/ul>\n<h2>Performance and Quality<\/h2>\n<p>Regarding the quality of the ONNX version, validation results using the <code>hf-internal-testing\/librispeech_asr_dummy<\/code> set have been reported. In the onnxruntime CPU EP environment, the 4-bit file recorded a 2.43% WER (2.31% when normalized). This indicates an improvement in accuracy compared to the 2.70% WER recorded by the previous 4.1 ONNX file.<\/p>\n<p>Furthermore, regarding the performance of the base model <code>granite-speech-5.0-2b<\/code>, a comparison with the predecessor model <code>granite-speech-4.1-2b<\/code> is shown in the table below.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th style=\"text-align: left;\">Test Set<\/th>\n<th style=\"text-align: center;\">PER 4.1-2b (\u2193)<\/th>\n<th style=\"text-align: center;\">PER 5.0-2b (\u2193)<\/th>\n<th style=\"text-align: center;\">Cap-F1 4.1-2b (\u2191)<\/th>\n<th style=\"text-align: center;\">Cap-F1 5.0-2b (\u2191)<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td style=\"text-align: left;\">LScln<\/td>\n<td style=\"text-align: center;\">25.70<\/td>\n<td style=\"text-align: center;\"><strong>23.20<\/strong><\/td>\n<td style=\"text-align: center;\">89.71<\/td>\n<td style=\"text-align: center;\"><strong>90.16<\/strong><\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\">LSoth<\/td>\n<td style=\"text-align: center;\">22.27<\/td>\n<td style=\"text-align: center;\"><strong>21.22<\/strong><\/td>\n<td style=\"text-align: center;\">91.26<\/td>\n<td style=\"text-align: center;\"><strong>91.65<\/strong><\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\">VoxPopuli<\/td>\n<td style=\"text-align: center;\">24.86<\/td>\n<td style=\"text-align: center;\"><strong>23.70<\/strong><\/td>\n<td style=\"text-align: center;\"><strong>95.35<\/strong><\/td>\n<td style=\"text-align: center;\">95.07<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\">Earnings-22<\/td>\n<td style=\"text-align: center;\">22.87<\/td>\n<td style=\"text-align: center;\"><strong>21.64<\/strong><\/td>\n<td style=\"text-align: center;\"><strong>95.19<\/strong><\/td>\n<td style=\"text-align: center;\">94.98<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\">CV-EN<\/td>\n<td style=\"text-align: center;\"><strong>9.13<\/strong><\/td>\n<td style=\"text-align: center;\">9.18<\/td>\n<td style=\"text-align: center;\">96.75<\/td>\n<td style=\"text-align: center;\"><strong>96.96<\/strong><\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\">CV-DE<\/td>\n<td style=\"text-align: center;\">3.66<\/td>\n<td style=\"text-align: center;\"><strong>3.65<\/strong><\/td>\n<td style=\"text-align: center;\">99.50<\/td>\n<td style=\"text-align: center;\"><strong>99.51<\/strong><\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\">CV-ES<\/td>\n<td style=\"text-align: center;\">11.61<\/td>\n<td style=\"text-align: center;\"><strong>11.46<\/strong><\/td>\n<td style=\"text-align: center;\"><strong>95.68<\/strong><\/td>\n<td style=\"text-align: center;\">95.62<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\">CV-FR<\/td>\n<td style=\"text-align: center;\"><strong>11.00<\/strong><\/td>\n<td style=\"text-align: center;\">11.43<\/td>\n<td style=\"text-align: center;\">97.25<\/td>\n<td style=\"text-align: center;\">97.20<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\">CV-PT<\/td>\n<td style=\"text-align: center;\">7.86<\/td>\n<td style=\"text-align: center;\"><strong>7.53<\/strong><\/td>\n<td style=\"text-align: center;\"><strong>98.51<\/strong><\/td>\n<td style=\"text-align: center;\">98.48<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>From this comparison table, it can be seen that the 5.0-2b model shows improvements in PER (Punctuation Error Rate), which indicates punctuation error rates, across many test sets. Specifically, it achieves lower PER than 4.1-2b on LScln, LSoth, VoxPopuli, Earnings-22, CV-ES, and CV-PT. On the other hand, for the PER of CV-EN, CV-DE, and CV-FR, 4.1-2b shows slightly lower or equivalent figures.<\/p>\n<p>Regarding Cap-F1 (Capitalization F1), which measures capitalization accuracy, 5.0-2b improves in items such as LScln, LSoth, CV-EN, CV-DE, and CV-PT, while 4.1-2b shows higher scores in VoxPopuli, Earnings-22, CV-ES, and CV-FR.<\/p>\n<h2>Strengths and Use Cases<\/h2>\n<p>This model is optimized for automatic speech recognition (ASR) and bidirectional automatic speech translation (AST) in English, French, German, Spanish, Portuguese, and Japanese. Features of the base model <code>ibm-granite\/granite-speech-5.0-2b<\/code> include improved transcription accuracy in short-form English ASR and enhanced robustness against noise, reverberation, and far-field microphone audio. It also features support for input language ID prompting in multilingual ASR, improved long-form and conversational speech translation, and an expanded keyword list size (up to 2000 words) in keyword-biased ASR and AST. By using the ONNX weights, in-browser inference can be performed via Transformers.js and WebGPU.<\/p>\n<p><!-- lmw:hardware --><\/p>\n<h2>Hardware Requirements<\/h2>\n<p><strong>Estimated requirements (calculated by Local Model Watch)<\/strong> \u2014 2.3B parameters (taken from the base model ibm-granite\/granite-speech-5.0-2b)<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Your VRAM<\/th>\n<th>Quantization<\/th>\n<th>File size<\/th>\n<th>Est. memory needed<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>8GB (RTX 4060 \/ 3060 Ti, etc.)<\/td>\n<td>BF16<\/td>\n<td>4.3GB<\/td>\n<td>5.2GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><strong>Inference engine support<\/strong> (architecture name matched against each project&#8217;s own model registry in its source code, checked 2026-11-17): llama.cpp: registered, vLLM: registered, MLX (mlx-lm): not registered. &#8220;Not registered&#8221; means the name is absent from that registry today, not that the model cannot run.<\/p>\n<p><em>Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model&#8217;s authors. Compare with other models in our <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-en\/\">VRAM quick reference<\/a>. What the quantization names mean: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">glossary<\/a>.<\/em><\/p>\n<p><!-- \/lmw:hardware --><\/p>\n<p><!-- lmw:runnability --><\/p>\n<h2>Can You Run It Locally?<\/h2>\n<p>The publisher distributes this model as ONNX.<\/p>\n<p><strong>License \u2014 <code>apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\n<p><em>Compiled by this site&#8217;s code from the published formats and the license field. License summaries are not legal advice \u2014 check the publisher&#8217;s original terms before relying on them.<\/em><\/p>\n<p><!-- \/lmw:runnability --><\/p>\n<p><!-- lmw:peers --><\/p>\n<h2>Recent Models in the Same Size Class<\/h2>\n<p><em>Models with <\/em><em>up to 4B<\/em><em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site&#8217;s estimates; licenses are as stated on the model cards.<\/em><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Model<\/th>\n<th>Parameters<\/th>\n<th>Smallest VRAM tier<\/th>\n<th>License<\/th>\n<th>Our article<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>ggml-org\/higgs-audio-v3-stt-GGUF<\/td>\n<td>2.7B<\/td>\n<td>4GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/higgs-audio-v3-stt-gguf\/\">higgs-audio-v3-stt-GGUF Speech-to-Text Model: 4GB+ VRAM, File List<\/a> (2026-11-17)<\/td>\n<\/tr>\n<tr>\n<td>FermionResearch\/Phonon-2<\/td>\n<td>627M<\/td>\n<td>4GB<\/td>\n<td>cc-by-4.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/01\/phonon-2-open-weight-english-asr-model\/\">Phonon-2 Speech Recognition Model: 4GB+ VRAM<\/a> (2026-09-30)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><!-- \/lmw:peers --><\/p>\n<p><!-- lmw:files --><\/p>\n<h2>Distributed Files<\/h2>\n<p><em>Weight files published in <a href=\"https:\/\/huggingface.co\/ibm-granite\/granite-speech-5.0-2b-ONNX\/tree\/main\">ibm-granite\/granite-speech-5.0-2b-ONNX<\/a>, listed by this site from the Hugging Face API. Sizes are the actual file sizes.<\/em><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>File<\/th>\n<th>Size<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><code>onnx\/audio_encoder_q4.onnx<\/code><\/td>\n<td>0MB<\/td>\n<\/tr>\n<tr>\n<td><code>onnx\/decoder_model_merged_q4f16.onnx<\/code><\/td>\n<td>0MB<\/td>\n<\/tr>\n<tr>\n<td><code>onnx\/embed_tokens_q4f16.onnx<\/code><\/td>\n<td>0MB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><!-- \/lmw:files --><\/p>\n<h2>How to Get It<\/h2>\n<p>Available from the Hugging Face repository <code>ibm-granite\/granite-speech-5.0-2b-ONNX<\/code>. It supports in-browser inference using Transformers.js (version 4.3 or later), and the official <code>granite-speech-webgpu<\/code> space is published as a demo. Specific usage code examples with Transformers.js are posted on the model card.<\/p>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/ibm-granite-switch-4-1-2\/\">IBM Releases Granite Switch 4.1: Modular LLMs with aLoRA and Speech<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Find models by VRAM<\/strong> (This model runs from the 8GB tier) \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-8gb-en\/\">Other models that run on a 8GB GPU<\/a><\/li>\n<li><strong>Formats this model is available in<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-onnx-en\/\">ONNX format guide and models<\/a><\/li>\n<li><strong>Learn about the publisher<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/publisher-ibm-en\/\">IBM (Granite): models, licenses and articles<\/a><\/li>\n<li><strong>Other models for the same task<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/models-by-task-en\/#task-audio\">Other audio models<\/a><\/li>\n<li><strong>How to read WER<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-benchmarks-en\/\">Benchmark glossary<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/huggingface.co\/ibm-granite\/granite-speech-5.0-2b-ONNX\">ibm-granite\/granite-speech-5.0-2b-ONNX<\/a><\/li>\n<li><a href=\"https:\/\/huggingface.co\/ibm-granite\/granite-speech-5.0-2b\">ibm-granite\/granite-speech-5.0-2b<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>IBM has released ONNX weights for Granite Speech 5.0 2B, enabling in-browser ASR and AST inference via Transformers.js and WebGPU.<\/p>\n","protected":false},"author":1,"featured_media":14027,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[203],"tags":[4300,3578,4302,1693,4304,1547,3018,1017],"class_list":["post-14028","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-image-video-and-audio","tag-granite-4-0-en","tag-ibm-en","tag-ibm-granite-en","tag-onnx-en","tag-transformers-js-en","tag-verified","tag-webgpu-en","tag--en"],"lang":"en","translations":{"en":14028,"ja":14026},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/14028","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=14028"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/14028\/revisions"}],"predecessor-version":[{"id":17986,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/14028\/revisions\/17986"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/14027"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=14028"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=14028"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=14028"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}