ggml-org Releases tinyLFM3-Omni-for-testing-gguf for CI

At a Glance
| Item | Value |
|---|---|
| Repository | ggml-org/tinyLFM3-Omni-for-testing-gguf |
| Published | 2026-11-06 |
| License | lfm1.0 |
| Formats | GGUF |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
ggml-org has released a lightweight repository, ggml-org/tinyLFM3-Omni-for-testing-gguf, on Hugging Face for the purpose of software testing and CI (Continuous Integration) verification in llama.cpp. This repository is a test model built by significantly reducing parameters and layer counts in order to run the code path of the multimodal model LiquidAI/LFM3-Omni-2B, developed by LiquidAI, in a lightweight environment.
The text and audio output from this model are meaningless data (gibberish) and cannot be used for actual chat or inference purposes. The sole purpose is to verify whether code paths in llama.cpp—such as TTS (Text-to-Speech), ASR (Automatic Speech Recognition), real-time audio input, and image input processing—function correctly under limited resources like CI environments.
Announcement Details
The characteristics of the model data published in this repository, as well as the specific modification procedures and configuration contents from the original model, are as follows:
Development/Verification Purpose and Usage Guide
This model is built for llama.cpp CI tests. Because it maintains the model structure of LFM3-Omni, it can execute the same execution paths as the real model, but its small model size makes it suitable for CI environments. Since it is not designed for practical inference or conversational use, users who want to use the actual model are directed to use the real model, ggml-org/LFM3-Omni-2B-GGUF (https://huggingface.co/ggml-org/LFM3-Omni-2B-GGUF).
Structural Changes from the Original Model
Slicing was performed from the original LiquidAI/LFM3-Omni-2B safetensors using the script slice_lfm3omni.py compliant with llama.cpp’s slice-model feature, and conversion to the GGUF format was carried out via convert_hf_to_gguf.py. The modifications for each component are as follows:
- Text model: Extracted only 4 layers (layers 0, 2, 3, 4) out of all 24 layers, reduced the hidden size from 2048 to 256, the number of attention heads from 16 to 2, the number of KV heads from 4 to 1, and the FFN (Feed-Forward Network) dimension from 7680 to 256.
- Audio encoder: Extracted 3 layers (layers 0, 2, 3) out of all 17 layers, and reduced the FFN dimension from 1792 to 256.
- Depthformer: Extracted only 1 layer out of all 3 layers, reduced the hidden size from 1024 to 128, the number of attention heads from 32 to 4, the number of KV heads from 8 to 1, and the FFN dimension from 2816 to 256.
- Audio codec: Due to loader-side specifications, all 8 layers are retained, but the FFN dimension is reduced from 2304 to 256.
- Vision encoder: Extracted 2 layers (layers 0, 2) out of all 14 layers, reduced the hidden size from 1024 to 256, the number of attention heads from 16 to 4, the number of KV heads from 8 to 2, the FFN dimension from 2560 to 256, and the projector dimension from 4096 to 256.
Additional Processing and Feature Adjustments
Using the finish_gguf.py script on the converted GGUF file, the following additional processing and adjustments were performed:
- Quantization implementation: Weight data is provided in Q2_K, Q4_K, and Q4_0 quantization formats.
- Addition of chat template: An
asrchat template forLFM3-Audio, which does not exist in the original model, was added. This allows testing the operation path of live transcription functionality (/v1/realtime?intent=transcription) for testing purposes.
The model specifies the lfm1.0 license and includes tags such as gguf, testing, and conversational.
Background
llama.cpp continues to be developed as an inference engine to run a diverse range of open-weight models locally. In accommodating the multimodal model LiquidAI/LFM3-Omni-2B developed by LiquidAI, processing paths for various modules such as TTS, ASR, and real-time audio and image inputs have been implemented in llama.cpp.
In open-source software development, automated CI (Continuous Integration) testing to guarantee the behavior of existing features during new feature additions or refactoring plays an extremely important role. However, loading and verifying the original model size (2B scale) as-is in a CI environment poses challenges such as storage and memory limitations and increased test execution times. Therefore, a test-specific lightweight model that maintains the same architecture and runs the same code paths as the real model, while operating lightweight and fast in a CI environment, has been required.
Impact on Local LLM Users
The main impacts and points to note for engineers and users running models locally regarding the newly released test repository are as follows:
Inability to Use for Actual Chat and Inference Purposes
The model provided in this repository is “sliced test data" with layer counts and parameter dimensions reduced to the absolute limit from the original model. Because all output results from the model are meaningless strings (gibberish), it cannot be used for practical purposes such as text generation, speech synthesis, or image understanding.
Availability and Usage Guide for the Real Model
For users who actually want to use the features of LFM3-Omni locally or in llama.cpp, it is clearly guided to use the real GGUF model ggml-org/LFM3-Omni-2B-GGUF (https://huggingface.co/ggml-org/LFM3-Omni-2B-GGUF).
Enhanced Verification of Multimodal Features in llama.cpp
With the introduction of this test model, automated testing of LFM3-Omni-related features in llama.cpp (TTS, ASR, real-time audio/image input) and the live transcription path (/v1/realtime?intent=transcription) via the added asr template can be constantly executed on CI. This is expected to maintain operational stability for these multimodal features in future llama.cpp version updates.
Mentions of Distribution Format and Future Release Policies
While the lfm1.0 license is specified for this model, this material does not contain mentions of future open-weight model release plans, the availability of existing models, or future distribution formats such as the balance between API provision and local execution.
Hardware Requirements
Estimated requirements (calculated by Local Model Watch)
| Your VRAM | Quantization | File size | Est. memory needed |
|---|---|---|---|
| 4GB (laptop iGPU / phone class) | Q2_K | 0.0GB | 0.0GB |
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.
Related Articles
- higgs-audio-v3-stt-GGUF Speech-to-Text Model: 4GB+ VRAM, File List
- Soprano-1.1-80M-GGUF Lightweight English TTS Model: File List
- VibeVoice-ASR-GGUF Speech Recognition and Diarization Model: 8GB+ VRAM
- Qwen3.5-4B-GGUF Vision-Language Model: 4GB+ VRAM
What to Read Next
- Find models by VRAM (This model runs from the 4GB tier) → Other models that run on a 8GB GPU
- Engines mentioned in this article → ggml / llama.cpp
- What Q2_K mean → Quantization and model-format glossary

