{"id":16452,"date":"2026-11-17T11:04:42","date_gmt":"2026-11-17T02:04:42","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/?p=16452"},"modified":"2026-11-17T11:14:08","modified_gmt":"2026-11-17T02:14:08","slug":"higgs-audio-v3-stt-gguf","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/higgs-audio-v3-stt-gguf\/","title":{"rendered":"higgs-audio-v3-stt-GGUF Speech-to-Text Model: 4GB+ VRAM, File List"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/huggingface.co\/ggml-org\/higgs-audio-v3-stt-GGUF\">ggml-org\/higgs-audio-v3-stt-GGUF<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-11-16<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>apache-2.0<\/td>\n<\/tr>\n<tr>\n<td>Formats<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-gguf-en\/\">GGUF<\/a><\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p><code>ggml-org\/higgs-audio-v3-stt-GGUF<\/code>, a GGUF format version of the Speech-to-Text (STT) model <code>bosonai\/higgs-audio-v3-stt<\/code> that takes audio data as input and outputs text, has been released.<\/p>\n<p>This model was converted into the GGUF format using ggml-org&#8217;s automatic conversion tool. Backed by speech recognition support in the llama.cpp project (Pull Request #32329), it can be run as a local server by executing the <code>llama serve<\/code> command from your terminal, or operated easily via the Llama app.<\/p>\n<h2>Specifications<\/h2>\n<p>Since this model is a quantized and GGUF-converted version, its model architecture and input specifications are based on the original model, <code>bosonai\/higgs-audio-v3-stt<\/code>. The specifications listed in the public documentation are as follows:<\/p>\n<ul>\n<li>Number of parameters: The total parameter count is 2.68B.<\/li>\n<li>Architecture: It adopts a custom configuration (HiggsAudio3Model) combining Whisper-Large-v3 for the encoder and Qwen3-1.7B for the decoder. The attention layers of Whisper-Large-v3 have been fine-tuned at the v2 stage.<\/li>\n<li>Input audio specifications: Supports 16kHz mono WAV format (resampling to 16kHz is required if the sampling rate differs).<\/li>\n<li>Thinking mode: Supports the use of a thinking mode to improve accuracy.<\/li>\n<li>License terms: Released under the Apache-2.0 license.<\/li>\n<\/ul>\n<h2>Performance and Quality<\/h2>\n<p>While this repository itself does not contain a benchmark score table, the model card for the original model, <code>bosonai\/higgs-audio-v3-stt<\/code>, reports updates to the training dataset and quality improvement efforts.<\/p>\n<p>The checkpoint of the original model was updated in June 2026, and the fine-tuning dataset was completely revamped. Specifically, in addition to the public training splits of AMI (IHM), VoxPopuli (en), SPGISpeech, LibriSpeech, TED-LIUM, and GigaSpeech, the public training split of Earnings22 (<code>sanchit-gandhi\/earnings22_split<\/code>) is used for training. Note that Earnings22 was trained after pre-filtering out data rows derived from audio sources included in the ESB and Open-ASR test sets.<\/p>\n<p>Furthermore, to suppress the output looping phenomenon that easily occurs in speech recognition tasks, in addition to the conventional word-level repetition limit settings, processing to automatically resolve phrase-level repetition loops has been introduced. This post-processing is implemented as a deterministic method and contributes to the stabilization of inference results.<\/p>\n<p>Regarding objective benchmark evaluation values, the publisher of the original model directs users to refer to the figures on the Open ASR Leaderboard, which is an independent verification environment. Scores listed in past model cards were based on older version checkpoints and evaluation environments, and are now deprecated.<\/p>\n<h2>Strengths and Use Cases<\/h2>\n<p>The <code>ggml-org\/higgs-audio-v3-stt-GGUF<\/code> model is designed to perform inference easily within the llama.cpp ecosystem while maintaining the speech recognition performance of the original model <code>bosonai\/higgs-audio-v3-stt<\/code>.<\/p>\n<p>The original model&#8217;s training dataset utilizes public training data from diverse domains such as AMI (meeting audio), VoxPopuli (English data from European Parliament speeches), SPGISpeech (financial speeches), LibriSpeech (audiobook readings), TED-LIUM (lecture audio), GigaSpeech (various podcasts and video audio), and Earnings22 (corporate earnings call audio). This enables it to handle English speech transcription across a wide range of situations, from clear read speech to everyday conversational speech and specialized business and financial presentations.<\/p>\n<p>Additionally, this model supports a thinking mode that leverages the language processing capabilities of the Qwen3 decoder. An approach to improve recognition accuracy by inserting a thinking process using <code>&lt;think&gt;<\/code> tags during inference is incorporated, and it is expected to output highly accurate transcripts that go beyond simple end-to-end speech recognition. Furthermore, because a deterministic phrase-level repetition loop elimination mechanism has been introduced, it suppresses the infinite loops of identical phrases often seen in conventional speech recognition models, making it suitable for applications requiring stable, long-form transcription.<\/p>\n<p>With this conversion to the GGUF format, speech recognition features can be used from simple llama.cpp execution binaries or desktop applications without having to individually set up a Python environment, PyTorch, or dedicated preprocessing libraries (such as <code>boson_multimodal<\/code>). This makes it a suitable choice for developers who want to launch a lightweight and fast speech recognition API server in a local environment, or engineers who want to build a self-contained meeting minute creation and speech transcription workflow on their devices.<\/p>\n<p><!-- lmw:hardware --><\/p>\n<h2>Hardware Requirements<\/h2>\n<p><strong>Estimated requirements (calculated by Local Model Watch)<\/strong> \u2014 2.7B parameters (taken from the base model bosonai\/higgs-audio-v3-stt)<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Your VRAM<\/th>\n<th>Quantization<\/th>\n<th>File size<\/th>\n<th>Est. memory needed<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>4GB (laptop iGPU \/ phone class)<\/td>\n<td>Q8_0<\/td>\n<td>2.0GB<\/td>\n<td>2.4GB<\/td>\n<\/tr>\n<tr>\n<td>8GB (RTX 4060 \/ 3060 Ti, etc.)<\/td>\n<td>BF16<\/td>\n<td>3.8GB<\/td>\n<td>4.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model&#8217;s authors. Compare with other models in our <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-en\/\">VRAM quick reference<\/a>. What the quantization names mean: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">glossary<\/a>.<\/em><\/p>\n<p><!-- \/lmw:hardware --><\/p>\n<p><!-- lmw:runnability --><\/p>\n<h2>Can You Run It Locally?<\/h2>\n<p>The publisher distributes this model as GGUF.<\/p>\n<p><strong>License \u2014 <code>apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\n<p><strong>Compression:<\/strong> the Q8_0 build measures 6.42 bits per weight \u2014 about 40% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\n<p><em>Compiled by this site&#8217;s code from the published formats and the license field. License summaries are not legal advice \u2014 check the publisher&#8217;s original terms before relying on them.<\/em><\/p>\n<p><!-- \/lmw:runnability --><\/p>\n<p><!-- lmw:peers --><\/p>\n<h2>Recent Models in the Same Size Class<\/h2>\n<p><em>Models with <\/em><em>up to 4B<\/em><em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site&#8217;s estimates; licenses are as stated on the model cards.<\/em><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Model<\/th>\n<th>Parameters<\/th>\n<th>Smallest VRAM tier<\/th>\n<th>License<\/th>\n<th>Our article<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>ibm-granite\/granite-speech-5.0-2b-ONNX<\/td>\n<td>2.3B<\/td>\n<td>8GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/ibm-granite-speech-5-0-2b-onnx\/\">granite-speech-5.0-2b-ONNX Speech Recognition Model: 8GB+ VRAM<\/a> (2026-11-17)<\/td>\n<\/tr>\n<tr>\n<td>FermionResearch\/Phonon-2<\/td>\n<td>627M<\/td>\n<td>4GB<\/td>\n<td>cc-by-4.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/01\/phonon-2-open-weight-english-asr-model\/\">Phonon-2 Speech Recognition Model: 4GB+ VRAM<\/a> (2026-09-30)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><!-- \/lmw:peers --><\/p>\n<p><!-- lmw:files --><\/p>\n<h2>Distributed Files<\/h2>\n<p><em>Weight files published in <a href=\"https:\/\/huggingface.co\/ggml-org\/higgs-audio-v3-stt-GGUF\/tree\/main\">ggml-org\/higgs-audio-v3-stt-GGUF<\/a>, listed by this site from the Hugging Face API. Sizes are the actual file sizes.<\/em><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>File<\/th>\n<th>Size<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><code>higgs-audio-v3-stt-BF16.gguf<\/code><\/td>\n<td>4.07GB<\/td>\n<\/tr>\n<tr>\n<td><code>higgs-audio-v3-stt-Q4_K_M.gguf<\/code><\/td>\n<td>1.28GB<\/td>\n<\/tr>\n<tr>\n<td><code>higgs-audio-v3-stt-Q8_0.gguf<\/code><\/td>\n<td>2.17GB<\/td>\n<\/tr>\n<tr>\n<td><code>mmproj-higgs-audio-v3-stt-BF16.gguf<\/code><\/td>\n<td>1.29GB<\/td>\n<\/tr>\n<tr>\n<td><code>mmproj-higgs-audio-v3-stt-Q8_0.gguf<\/code><\/td>\n<td>696MB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><!-- \/lmw:files --><\/p>\n<h2>How to Get It<\/h2>\n<p>This model is distributed in the GGUF format on the Hugging Face repository <code>ggml-org\/higgs-audio-v3-stt-GGUF<\/code>. It is not a gated model requiring prior agreement to terms of use or access requests, so anyone can directly download and start using it.<\/p>\n<p>The distributor guides users to run it in the desktop application Llama app (<a href=\"https:\/\/llama.app\">https:\/\/llama.app<\/a>), and it can also be quickly started as a local server by running the following command from the terminal:<\/p>\n<pre><code class=\"language-bash\">llama serve -hf ggml-org\/higgs-audio-v3-stt-GGUF\n<\/code><\/pre>\n<p>This GGUF model was generated using the automatic conversion tool provided by ggml-org (<a href=\"https:\/\/github.com\/ggml-org\/convert\">https:\/\/github.com\/ggml-org\/convert<\/a>). Because it operates using speech recognition model support on the llama.cpp side (Pull Request #32329), please use the latest version of the llama.cpp build that supports this feature when using it.<\/p>\n<p>If you wish to use the original PyTorch implementation or pre-conversion weights, please refer to the original model repository <code>bosonai\/higgs-audio-v3-stt<\/code>. Running the original model directly requires specifying <code>trust_remote_code=True<\/code> to load the custom architecture (HiggsAudio3Model), the <code>boson_multimodal<\/code> library for audio preprocessing, and resampling processing to 16kHz mono WAV format.<\/p>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/soprano-1-1-80m-gguf-released\/\">Soprano-1.1-80M-GGUF Lightweight English TTS Model: File List<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/run-resemble-ai-chatterbox-turbo-locally-gguf\/\">chatterbox-turbo-GGUF Text-to-Speech Model: 4GB+ VRAM, File List<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/chatterbox-multilingual-v3-gguf-2\/\">chatterbox-GGUF 23-Language Multilingual TTS Model: 4GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/voxtral-mini-4b-realtime-2602-gguf-2\/\">Voxtral-Mini-4B-Realtime-2602-GGUF Real-time ASR Model: 4GB+ VRAM<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Find models by VRAM<\/strong> (This model runs from the 4GB tier) \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-8gb-en\/\">Other models that run on a 8GB GPU<\/a><\/li>\n<li><strong>What Q8_0 mean and where to get this model<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-gguf-en\/\">GGUF format guide and models<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">Quantization and model-format glossary<\/a><\/li>\n<li><strong>Other models for the same task<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/models-by-task-en\/#task-audio\">Other audio models<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/huggingface.co\/ggml-org\/higgs-audio-v3-stt-GGUF\">https:\/\/huggingface.co\/ggml-org\/higgs-audio-v3-stt-GGUF<\/a><\/li>\n<li><a href=\"https:\/\/huggingface.co\/bosonai\/higgs-audio-v3-stt\">https:\/\/huggingface.co\/bosonai\/higgs-audio-v3-stt<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>The GGUF version of bosonai\/higgs-audio-v3-stt is now available, enabling local speech recognition via llama.cpp.<\/p>\n","protected":false},"author":1,"featured_media":16451,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[203],"tags":[5090,161,5092,163,5094,1073,2779,1547,1017],"class_list":["post-16452","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-image-video-and-audio","tag-bosonai-higgs-audio-v3-stt-en","tag-ggml-en","tag-ggml-org-higgs-audio-v3-stt-gguf-en","tag-gguf-en","tag-higgs-audio-v3-stt-en","tag-llama-cpp-en","tag-speech-to-text-en","tag-verified","tag--en"],"lang":"en","translations":{"en":16452,"ja":16450},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/16452","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=16452"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/16452\/revisions"}],"predecessor-version":[{"id":18128,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/16452\/revisions\/18128"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/16451"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=16452"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=16452"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=16452"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}