Voxtral-Mini-4B-Realtime-2602-GGUF Real-time ASR Model: 4GB+ VRAM

Voxtral-Mini-4B-Realtime-2602-GGUF Real-time ASR Model: 4GB+ VRAM

At a Glance

Item Value
Repository ggml-org/Voxtral-Mini-4B-Realtime-2602-GGUF
Family guide Ministral-3 guide (2 articles)
Publisher guide Mistral AI: models and licenses
Published 2026-10-16
License apache-2.0
Formats GGUF
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

ggml-org/Voxtral-Mini-4B-Realtime-2602-GGUF is a quantized model in GGUF format based on mistralai/Voxtral-Mini-4B-Realtime-2602. This model was developed for streaming Automatic Speech Recognition (ASR), converting voice inputs into text in real time. It is designed to allow sequential transcription through streaming sessions of audio data, making it suitable for applications requiring low latency and real-time performance. This repository provides the quantized version automatically converted by ggml-org.

Specifications

Technical specifications and license information for this model are as follows:

  • Architecture: VoxtralRealtimeForConditionalGeneration
  • Pipeline: automatic-speech-recognition
  • Distribution Format: GGUF
  • License: apache-2.0
  • Supported Languages: English, French, Spanish, German, Russian, Chinese, Japanese, Italian, Portuguese, Dutch, Arabic, Hindi, Korean
  • Features: Designed to support conversational use cases as well.

Performance and Quality

As an evaluation of the performance of the source model Voxtral Mini 4B Realtime 2602, comparison results using the Fleurs benchmark are provided. This benchmark compares the accuracy of the offline Voxtral Mini Transcribe 2.0 with real-time models configured with different delay settings.

→ Scroll horizontally to see all columns

Model Delay AVG Arabic German English Spanish French Hindi Italian Dutch Portuguese Chinese Japanese Korean Russian
Voxtral Mini Transcribe 2.0 Offline 5.90% 13.54% 3.54% 3.32% 2.63% 4.32% 10.33% 2.17% 4.78% 3.56% 7.30% 4.14% 12.29% 4.75%
Voxtral Mini 4B Realtime 2602 480 ms 8.72% 22.53% 6.19% 4.90% 3.31% 6.42% 12.88% 3.27% 7.07% 5.03% 10.45% 9.59% 15.74% 6.02%
160 ms 12.60% 24.33% 9.50% 6.46% 5.34% 9.75% 15.28% 5.59% 11.39% 10.01% 17.67% 19.17% 19.81% 9.53%
240 ms 10.80% 23.95% 8.15% 5.91% 4.59% 8.00% 14.26% 4.41% 9.23% 7.51% 13.84% 15.17% 17.56% 7.87%
960 ms 7.70% 20.32% 4.87% 4.34% 2.98% 5.68% 11.82% 2.46% 6.76% 4.57% 8.99% 6.80% 14.90% 5.56%
2400 ms 6.73% 14.71% 4.15% 4.05% 2.71% 5.23% 10.73% 2.37% 5.91% 3.93% 8.48% 5.50% 14.30% 5.41%

From this table, the offline model Voxtral Mini Transcribe 2.0 shows the lowest error rate with an average (AVG) of 5.90%. For the real-time model, error rates tend to decrease as the configured delay increases. For instance, the average error rate is 12.60% at a 160ms delay, but improves to 6.73% when the delay is increased to 2400ms.

According to the description in the model card, Voxtral Mini 4B Realtime 2602 achieves an accuracy comparable to major offline models, and is reported to show significant improvements in accuracy compared to existing open-source real-time solutions.

Strengths and Use Cases

This model specializes in real-time automatic speech recognition (ASR) that sequentially transcribes streaming audio input. The model card emphasizes its ability to continuously transcribe with low latency through streaming audio sessions.

Specific anticipated use cases include conversational applications requiring real-time performance and live transcription of audio data. Additionally, given the conversational tag, it is designed to be suitable for interactive audio processing. It supports multiple languages, including Japanese, allowing for deployment across a wide range of linguistic environments.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch) — 4.4B parameters (taken from the base model mistralai/Voxtral-Mini-4B-Realtime-2602)

Your VRAM Quantization File size Est. memory needed
4GB (laptop iGPU / phone class) Q4_K_M 2.4GB 2.9GB
8GB (RTX 4060 / 3060 Ti, etc.) BF16 6.4GB 7.7GB

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

The publisher distributes this model as GGUF.

License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.

Compression: the Q4_K_M build measures 4.65 bits per weight — about 29% the size of the original 16-bit weights, calculated by this site from the actual file sizes.

Compiled by this site’s code from the published formats and the license field. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Recent Models in the Same Size Class

Models with 4–15B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.

→ Scroll horizontally to see all columns

Model Parameters Smallest VRAM tier License Our article
ggml-org/VibeVoice-ASR-GGUF 8.3B 8GB mit VibeVoice-ASR-GGUF Speech Recognition and Diarization Model: 8GB+ VRAM (2026-11-17)
Edge0/Audio8-ASR-Infinite 4.1B 12GB apache-2.0 Audio8-ASR-Infinite Speech Recognition Model: 12GB+ VRAM, File List (2026-09-24)

Distributed Files

Weight files published in ggml-org/Voxtral-Mini-4B-Realtime-2602-GGUF, listed by this site from the Hugging Face API. Sizes are the actual file sizes.

File Size
mmproj-Voxtral-Mini-4B-Realtime-2602-BF16.gguf 1.99GB
mmproj-Voxtral-Mini-4B-Realtime-2602-Q8_0.gguf 1.06GB
Voxtral-Mini-4B-Realtime-2602-BF16.gguf 6.88GB
Voxtral-Mini-4B-Realtime-2602-Q4_K_M.gguf 2.55GB
Voxtral-Mini-4B-Realtime-2602-Q8_0.gguf 3.66GB

How to Get It

This model is distributed in GGUF format. It is available from the following repository:

  • Repository: ggml-org/Voxtral-Mini-4B-Realtime-2602-GGUF

According to the source code description, serving using llama.cpp is possible. It indicates that it can be launched with the following command:

llama serve -hf ggml-org/Voxtral-Mini-4B-Realtime-2602-GGUF

Related Articles

What to Read Next

Sources