Rumble-2-Flash-GGUF 45-Language ASR Model: 4GB+ VRAM, File List

Sample outputs are available on the model card.
At a Glance
| Item | Value |
|---|---|
| Repository | AtomicChat/Rumble-2-Flash-GGUF |
| Published | 2026-11-19 |
| License | apache-2.0 |
| Formats | GGUF |
| Source type | Unverified (not confirmed by a primary source) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
It is reported that the AtomicChat team has released GGUF format files for the speech recognition model “Rumble-2-Flash". This model is designed to take audio data as input and transcribe its contents into text (speech recognition). Note that this information has not been officially confirmed at this time.
This model was created by the team developing “Atomic Chat", an app for running open-weight models locally and offline. The base model is Reson8/Rumble-2-Flash, and it has been quantized for use with llama.cpp. The model consists of an 18-layer audio encoder and a 14-layer decoder.
Specifications
The specifications listed in the documentation are as follows:
- Architecture: 18-layer audio encoder and 14-layer decoder
- Base model:
Qwen3-ASR-0.6B - Number of parameters: 0.5B
- Supported languages: 45 languages
- License: Apache 2.0
- Distribution format: GGUF
Performance and Quality
The model card shows performance comparison data across different quantization levels. The evaluation uses datasets such as LibriSpeech, Multilingual LibriSpeech (MLS), and FLEURS. Performance metrics used are WER (Word Error Rate, lower is better) and CER (Character Error Rate, lower is better) for transcriptions.
The following table shows a comparison of performance across each quantized file. Note that these figures are results measured using fixes applied to llama.cpp.
→ Scroll horizontally to see all columns
| File | GB on disk | bpw | LS clean | LS other | MLS DE | MLS ES | MLS FR | FLEURS ZH |
|---|---|---|---|---|---|---|---|---|
Rumble-2-Flash-BF16.gguf |
0.65 | 16.0 | 2.09 | 4.70 | 6.85 | 4.57 | 6.57 | 10.38 |
Rumble-2-Flash-Q8_0.gguf |
0.35 | 8.50 | 2.11 | 4.70 | 6.79 | 4.63 | 6.68 | 10.31 |
Rumble-2-Flash-AD-Q5_K_M.gguf |
0.28 | 6.94 | 2.09 | 4.78 | 6.83 | 4.51 | 6.60 | 10.52 |
stock Q6_K (baseline, not shipped) |
0.27 | 6.56 | 2.20 | 4.64 | 6.84 | 4.51 | 6.66 | 10.44 |
our AD-Q4_K_M (tried, not shipped) |
0.26 | 6.48 | 2.22 | 4.81 | 7.05 | 4.74 | 6.88 | 10.54 |
stock Q5_K_M (baseline, not shipped) |
0.24 | 5.92 | 2.14 | 4.89 | 7.00 | 4.57 | 6.87 | 10.76 |
stock Q4_K_M (baseline, not shipped) |
0.22 | 5.32 | 2.42 | 5.21 | 7.62 | 5.14 | 7.47 | 11.99 |
As a result of the comparison, it can be seen that the recommended AD-Q5_K_M maintains performance within or extremely close to BF16 (original precision) across most test sets. On the other hand, quantizations aimed at higher compression rates, such as Q4_K_M, tend to show a decrease in accuracy across all test sets. In particular, for Mandarin CER, a degradation of up to 1.6 points is reported with Q4_K_M.
Additionally, it has been reported that the standard implementation (stock) of llama.cpp passes zero-padding at the end of audio clips to the model as sound, causing shorter clips to transcribe words that were not actually spoken. The figures in this documentation include a fix applied to resolve this issue. With this fix, llama.cpp is reported to exhibit performance that matches the Transformers reference model within the margin of noise.
Strengths and Use Cases
This model specializes in Automatic Speech Recognition (ASR), converting audio data into text. It supports as many as 45 languages and is intended for use in a wide range of linguistic environments. Specifically, it is suited for the following use cases:
- Multilingual audio transcription
- Converting voice data into text
- Local offline speech recognition
Furthermore, it is built to run on llama.cpp, allowing for detailed transcription controls via chat_template_kwargs, such as a digits setting to output numbers in numeric format, retaining or removing fillers like “um" and “ah", and condensing repeated words.
Hardware Requirements
Estimated requirements (calculated by Local Model Watch) — 509M parameters (taken from the base model Reson8/Rumble-2-Flash)
| Your VRAM | Quantization | File size | Est. memory needed |
|---|---|---|---|
| 4GB (laptop iGPU / phone class) | BF16 | 0.6GB | 0.7GB |
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.
Can You Run It Locally?
The publisher distributes this model as GGUF.
License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.
Compiled by this site’s code from the published formats and the license field. License summaries are not legal advice — check the publisher’s original terms before relying on them.
Recent Models in the Same Size Class
Models with up to 4B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.
→ Scroll horizontally to see all columns
| Model | Parameters | Smallest VRAM tier | License | Our article |
|---|---|---|---|---|
| ibm-granite/granite-speech-5.0-2b-ONNX | 2.3B | 8GB | apache-2.0 | granite-speech-5.0-2b-ONNX Speech Recognition Model: 8GB+ VRAM (2026-11-17) |
| ggml-org/higgs-audio-v3-stt-GGUF | 2.7B | 4GB | apache-2.0 | higgs-audio-v3-stt-GGUF Speech-to-Text Model: 4GB+ VRAM, File List (2026-11-17) |
| FermionResearch/Phonon-2 | 627M | 4GB | cc-by-4.0 | Phonon-2 Speech Recognition Model: 4GB+ VRAM (2026-09-30) |
Distributed Files
Weight files published in AtomicChat/Rumble-2-Flash-GGUF, listed by this site from the Hugging Face API. Sizes are the actual file sizes.
| File | Size |
|---|---|
mmproj-Rumble-2-Flash-F16.gguf |
379MB |
mmproj-Rumble-2-Flash-Q8_0.gguf |
214MB |
Rumble-2-Flash-AD-Q5_K_M.gguf |
283MB |
Rumble-2-Flash-BF16.gguf |
648MB |
Rumble-2-Flash-Q8_0.gguf |
346MB |
How to Get It
This model is distributed in GGUF format. Using it requires two files: the main model file (.gguf) and the mmproj audio encoder file. The recommended combination is the pair of Rumble-2-Flash-AD-Q5_K_M.gguf and mmproj-Rumble-2-Flash-Q8_0.gguf.
You can run it using llama.cpp with a command like the following:
llama-server -m Rumble-2-Flash-AD-Q5_K_M.gguf --mmproj mmproj-Rumble-2-Flash-Q8_0.gguf -ngl 99
When using the command-line interface (llama-mtmd-cli), it is recommended to run it with the --jinja flag to correctly apply the embedded template.
llama-mtmd-cli -m Rumble-2-Flash-AD-Q5_K_M.gguf --mmproj mmproj-Rumble-2-Flash-Q8_0.gguf --jinja --audio audio.wav -p "lang:en" --temp 0
The repository is available on Hugging Face at AtomicChat/Rumble-2-Flash-GGUF.
Related Articles
- Aion-1.0-Instruct-2.3B-GGUF 2.3B Text Generation Model: 4GB+ VRAM
- Maxwell-V1-Flash-GGUF 35B MoE Model: 24GB+ VRAM
What to Read Next
- Find models by VRAM (This model runs from the 4GB tier) → Other models that run on a 8GB GPU
- Formats this model is available in → GGUF format guide and models
- Other models for the same task → Other audio models
- How to read WER, CER → Benchmark glossary
Sources
This article contains unverified information. We will append an update note once it is confirmed by a primary source.
