VibeVoice-ASR-GGUF Speech Recognition and Diarization Model: 8GB+ VRAM

VibeVoice-ASR-GGUF Speech Recognition and Diarization Model: 8GB+ VRAM

At a Glance

Item Value
Repository ggml-org/VibeVoice-ASR-GGUF
Publisher guide Microsoft: models and licenses
Published 2026-11-16
License mit
Formats GGUF
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

ggml-org has released “ggml-org/VibeVoice-ASR-GGUF", a GGUF format conversion of Microsoft’s speech recognition model “VibeVoice-ASR". It is a Speech-to-Text model that takes audio data as input and outputs structured transcriptions along with “who spoke (speaker identification)", “when (timestamps)", and “what was said". It is compatible with the llama.cpp ecosystem, enabling easy execution in local environments.

Specifications

  • Base Model: microsoft/VibeVoice-ASR-HF
  • Architecture: VibeVoiceAsrForConditionalGeneration (model_type: vibevoice_asr)
  • Supported Languages: Over 50 languages including English, Chinese, Japanese, Spanish, German, and French (with native support for code-switching between multiple languages)
  • Audio Processing Capacity: Single-pass processing of up to 60 minutes of continuous audio within a 64K token length (also supports chunked processing in 60-second units, 24kHz sampling rate)
  • Licensing Terms: MIT License (both the base model and GGUF version are provided under the MIT License and can be used by retaining the specified copyright and permission notices)

Performance and Quality

Evaluation results on the Open ASR Leaderboard published in the model card for the original model “microsoft/VibeVoice-ASR-HF" are as follows (these figures are publicly reported values for the original model). WER (Word Error Rate) is an index indicating the word-level error rate of transcriptions, where lower numbers mean better performance.

Dataset WER (%)
ami_test 17.20
earnings22_test 13.17
gigaspeech_test 9.67
librispeech_test.clean 2.20
librispeech_test.other 5.51
spgispeech_test 3.80
tedlium_test 2.57
voxpopuli_test 8.01
Average 7.77
RTFx 51.80

According to the published measurement results, the average WER for the test dataset achieves 7.77%. While it shows low error rates on datasets such as “librispeech_test.clean" (2.20%) and “tedlium_test" (2.57%), error rates tend to be higher on meeting audio like “ami_test" (17.20%) and financial data like “earnings22_test" (13.17%). Additionally, RTFx, an index of processing speed, is reported as 51.80.

Strengths and Use Cases

VibeVoice-ASR is a model that demonstrates high effectiveness in transcribing long conversational audio and analyzing dialogues involving multiple participants. Based on the specifications of the original model “microsoft/VibeVoice-ASR-HF", it has strengths in the following uses and scenarios:

Batch Processing of Up to 60 Minutes of Long Audio in a Single Pass

Many traditional speech recognition models process audio by slicing it into short segments, which poses the challenge of breaking context and speaker information. VibeVoice-ASR leverages a 64K token context length to process up to 60 minutes of continuous audio in a single pass. This makes it possible to transcribe long audio content such as podcasts, lengthy business meetings, interviews, and lectures with high accuracy while maintaining consistency in overall context and speaker tracking.

Integrated Output of Speaker Identification, Timestamps, and Utterance Content

VibeVoice-ASR performs speaker diarization (who spoke) and timestamping (when they spoke) simultaneously along with automatic speech recognition (ASR). The model’s output is structured JSON-like text, obtained in a format that summarizes start times, end times, speaker IDs, and utterance text. This allows for creating minutes of meetings and generating subtitle data in a one-stop manner without combining separate diarization or alignment tools.

Correction of Recognition Accuracy via Custom Hotwords

By providing proper nouns such as names of people, companies, or technical terms, or contextual information to the model as prompts (Hotwords), recognition accuracy in specific domains can be significantly improved. Published materials show an example where even in a case where German-accented English audio was misrecognized without a prompt, the brand name could be accurately transcribed by attaching the context prompt “About VibeVoice".

Multilingual Recognition of Over 50 Languages and Support for Code-Switching

It supports over 50 languages, including Japanese, English, Chinese, Spanish, French, and German. There is no need to specify the language in advance, as the model automatically recognizes the audio. Furthermore, because it natively supports code-switching where languages change within the same utterance or midway through a conversation, it is also suitable for processing international online meetings and natural dialogue audio where multiple languages are mixed.

Flexible Execution in Local Environments via GGUF Format

This model, “ggml-org/VibeVoice-ASR-GGUF", has been converted to the GGUF format through ggml-org’s automated conversion pipeline. Support for the llama.cpp ecosystem makes it possible to execute speech recognition directly on your device using lightweight binaries, without having to spin up Python environments or heavy deep learning frameworks.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch) — 8.3B parameters (taken from the base model microsoft/VibeVoice-ASR-HF)

Your VRAM Quantization File size Est. memory needed
8GB (RTX 4060 / 3060 Ti, etc.) Q4_K_M 4.4GB 5.2GB
12GB (RTX 4070 / 3060 12GB, etc.) Q8_0 7.5GB 9.1GB
24GB (RTX 4090 / 3090, etc.) BF16 14.2GB 17.0GB

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

The publisher distributes this model as GGUF.

License — mit (Commercial use allowed): Permits commercial use, modification and redistribution, provided the copyright notice and license text are retained.

Compression: the Q4_K_M build measures 4.54 bits per weight — about 28% the size of the original 16-bit weights, calculated by this site from the actual file sizes.

Compiled by this site’s code from the published formats and the license field. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Recent Models in the Same Size Class

Models with 4–15B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.

→ Scroll horizontally to see all columns

Model Parameters Smallest VRAM tier License Our article
mistralai/Voxtral-Mini-4B-Realtime-Arabic 4.4B 4GB — Voxtral-Mini-4B-Realtime-Arabic Speech Recognition Model: 4GB+ VRAM (2026-11-17)
ggml-org/Voxtral-Mini-4B-Realtime-2602-GGUF 4.4B 4GB apache-2.0 Voxtral-Mini-4B-Realtime-2602-GGUF Real-time ASR Model: 4GB+ VRAM (2026-11-17)
Edge0/Audio8-ASR-Infinite 4.1B 12GB apache-2.0 Audio8-ASR-Infinite Speech Recognition Model: 12GB+ VRAM, File List (2026-09-24)

Distributed Files

Weight files published in ggml-org/VibeVoice-ASR-GGUF, listed by this site from the Hugging Face API. Sizes are the actual file sizes.

File Size
mmproj-VibeVoice-ASR-BF16.gguf 1.43GB
mmproj-VibeVoice-ASR-Q8_0.gguf 840MB
VibeVoice-ASR-BF16.gguf 15.24GB
VibeVoice-ASR-Q4_K_M.gguf 4.68GB
VibeVoice-ASR-Q8_0.gguf 8.10GB

How to Get It

This model is distributed in GGUF format on the Hugging Face repository “ggml-org/VibeVoice-ASR-GGUF". There are no gates requiring prior access requests or license agreements for use, and anyone can directly download and use it.

Using with Llama app

If you want to easily use it in a GUI environment, you can run the model using the Llama app ( https://llama.app ), which has official support guidance.

Running from the Terminal

To launch it as a server from the command line, use the llama.cpp tool and run the following command:

llama serve -hf ggml-org/VibeVoice-ASR-GGUF

This model was automatically converted using ggml-org’s conversion tool (convert), and support as a Speech-to-Text model in llama.cpp is progressing. Detailed implementation status and PR information can be checked in llama.cpp Pull Request #32329.

Note that if you want to use the original model weights or implementations using the Transformers library, please refer to the base model’s repository, “microsoft/VibeVoice-ASR-HF".

Related Articles

What to Read Next

Sources