{"id":15307,"date":"2026-11-17T11:03:39","date_gmt":"2026-11-17T02:03:39","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/?p=15307"},"modified":"2026-11-17T11:13:14","modified_gmt":"2026-11-17T02:13:14","slug":"ggml-org-releases-tinylfm3-omni-for-testing-gguf","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/ggml-org-releases-tinylfm3-omni-for-testing-gguf\/","title":{"rendered":"ggml-org Releases tinyLFM3-Omni-for-testing-gguf for CI"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/huggingface.co\/ggml-org\/tinyLFM3-Omni-for-testing-gguf\">ggml-org\/tinyLFM3-Omni-for-testing-gguf<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-11-06<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>lfm1.0<\/td>\n<\/tr>\n<tr>\n<td>Formats<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-gguf-en\/\">GGUF<\/a><\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>ggml-org has released a lightweight repository, <code>ggml-org\/tinyLFM3-Omni-for-testing-gguf<\/code>, on Hugging Face for the purpose of software testing and CI (Continuous Integration) verification in llama.cpp. This repository is a test model built by significantly reducing parameters and layer counts in order to run the code path of the multimodal model <code>LiquidAI\/LFM3-Omni-2B<\/code>, developed by LiquidAI, in a lightweight environment.<\/p>\n<p>The text and audio output from this model are meaningless data (gibberish) and cannot be used for actual chat or inference purposes. The sole purpose is to verify whether code paths in llama.cpp\u2014such as TTS (Text-to-Speech), ASR (Automatic Speech Recognition), real-time audio input, and image input processing\u2014function correctly under limited resources like CI environments.<\/p>\n<h2>Announcement Details<\/h2>\n<p>The characteristics of the model data published in this repository, as well as the specific modification procedures and configuration contents from the original model, are as follows:<\/p>\n<h3>Development\/Verification Purpose and Usage Guide<\/h3>\n<p>This model is built for llama.cpp CI tests. Because it maintains the model structure of LFM3-Omni, it can execute the same execution paths as the real model, but its small model size makes it suitable for CI environments. Since it is not designed for practical inference or conversational use, users who want to use the actual model are directed to use the real model, <code>ggml-org\/LFM3-Omni-2B-GGUF<\/code> (<a href=\"https:\/\/huggingface.co\/ggml-org\/LFM3-Omni-2B-GGUF\"><a href=\"https:\/\/huggingface.co\/ggml-org\/LFM3-Omni-2B-GGUF\">https:\/\/huggingface.co\/ggml-org\/LFM3-Omni-2B-GGUF<\/a><\/a>).<\/p>\n<h3>Structural Changes from the Original Model<\/h3>\n<p>Slicing was performed from the original <code>LiquidAI\/LFM3-Omni-2B<\/code> safetensors using the script <code>slice_lfm3omni.py<\/code> compliant with llama.cpp&#8217;s <code>slice-model<\/code> feature, and conversion to the GGUF format was carried out via <code>convert_hf_to_gguf.py<\/code>. The modifications for each component are as follows:<\/p>\n<ul>\n<li><strong>Text model<\/strong>: Extracted only 4 layers (layers 0, 2, 3, 4) out of all 24 layers, reduced the hidden size from 2048 to 256, the number of attention heads from 16 to 2, the number of KV heads from 4 to 1, and the FFN (Feed-Forward Network) dimension from 7680 to 256.<\/li>\n<li><strong>Audio encoder<\/strong>: Extracted 3 layers (layers 0, 2, 3) out of all 17 layers, and reduced the FFN dimension from 1792 to 256.<\/li>\n<li><strong>Depthformer<\/strong>: Extracted only 1 layer out of all 3 layers, reduced the hidden size from 1024 to 128, the number of attention heads from 32 to 4, the number of KV heads from 8 to 1, and the FFN dimension from 2816 to 256.<\/li>\n<li><strong>Audio codec<\/strong>: Due to loader-side specifications, all 8 layers are retained, but the FFN dimension is reduced from 2304 to 256.<\/li>\n<li><strong>Vision encoder<\/strong>: Extracted 2 layers (layers 0, 2) out of all 14 layers, reduced the hidden size from 1024 to 256, the number of attention heads from 16 to 4, the number of KV heads from 8 to 2, the FFN dimension from 2560 to 256, and the projector dimension from 4096 to 256.<\/li>\n<\/ul>\n<h3>Additional Processing and Feature Adjustments<\/h3>\n<p>Using the <code>finish_gguf.py<\/code> script on the converted GGUF file, the following additional processing and adjustments were performed:<\/p>\n<ul>\n<li><strong>Quantization implementation<\/strong>: Weight data is provided in Q2_K, Q4_K, and Q4_0 quantization formats.<\/li>\n<li><strong>Addition of chat template<\/strong>: An <code>asr<\/code> chat template for <code>LFM3-Audio<\/code>, which does not exist in the original model, was added. This allows testing the operation path of live transcription functionality (<code>\/v1\/realtime?intent=transcription<\/code>) for testing purposes.<\/li>\n<\/ul>\n<p>The model specifies the <code>lfm1.0<\/code> license and includes tags such as <code>gguf<\/code>, <code>testing<\/code>, and <code>conversational<\/code>.<\/p>\n<h2>Background<\/h2>\n<p>llama.cpp continues to be developed as an inference engine to run a diverse range of open-weight models locally. In accommodating the multimodal model <code>LiquidAI\/LFM3-Omni-2B<\/code> developed by LiquidAI, processing paths for various modules such as TTS, ASR, and real-time audio and image inputs have been implemented in llama.cpp.<\/p>\n<p>In open-source software development, automated CI (Continuous Integration) testing to guarantee the behavior of existing features during new feature additions or refactoring plays an extremely important role. However, loading and verifying the original model size (2B scale) as-is in a CI environment poses challenges such as storage and memory limitations and increased test execution times. Therefore, a test-specific lightweight model that maintains the same architecture and runs the same code paths as the real model, while operating lightweight and fast in a CI environment, has been required.<\/p>\n<h2>Impact on Local LLM Users<\/h2>\n<p>The main impacts and points to note for engineers and users running models locally regarding the newly released test repository are as follows:<\/p>\n<h3>Inability to Use for Actual Chat and Inference Purposes<\/h3>\n<p>The model provided in this repository is &#8220;sliced test data&#8221; with layer counts and parameter dimensions reduced to the absolute limit from the original model. Because all output results from the model are meaningless strings (gibberish), it cannot be used for practical purposes such as text generation, speech synthesis, or image understanding.<\/p>\n<h3>Availability and Usage Guide for the Real Model<\/h3>\n<p>For users who actually want to use the features of LFM3-Omni locally or in llama.cpp, it is clearly guided to use the real GGUF model <code>ggml-org\/LFM3-Omni-2B-GGUF<\/code> (<a href=\"https:\/\/huggingface.co\/ggml-org\/LFM3-Omni-2B-GGUF\"><a href=\"https:\/\/huggingface.co\/ggml-org\/LFM3-Omni-2B-GGUF\">https:\/\/huggingface.co\/ggml-org\/LFM3-Omni-2B-GGUF<\/a><\/a>).<\/p>\n<h3>Enhanced Verification of Multimodal Features in llama.cpp<\/h3>\n<p>With the introduction of this test model, automated testing of LFM3-Omni-related features in llama.cpp (TTS, ASR, real-time audio\/image input) and the live transcription path (<code>\/v1\/realtime?intent=transcription<\/code>) via the added <code>asr<\/code> template can be constantly executed on CI. This is expected to maintain operational stability for these multimodal features in future llama.cpp version updates.<\/p>\n<h3>Mentions of Distribution Format and Future Release Policies<\/h3>\n<p>While the <code>lfm1.0<\/code> license is specified for this model, this material does not contain mentions of future open-weight model release plans, the availability of existing models, or future distribution formats such as the balance between API provision and local execution.<\/p>\n<p><!-- lmw:hardware --><\/p>\n<h2>Hardware Requirements<\/h2>\n<p><strong>Estimated requirements (calculated by Local Model Watch)<\/strong><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Your VRAM<\/th>\n<th>Quantization<\/th>\n<th>File size<\/th>\n<th>Est. memory needed<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>4GB (laptop iGPU \/ phone class)<\/td>\n<td>Q2_K<\/td>\n<td>0.0GB<\/td>\n<td>0.0GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model&#8217;s authors. Compare with other models in our <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-en\/\">VRAM quick reference<\/a>. What the quantization names mean: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">glossary<\/a>.<\/em><\/p>\n<p><!-- \/lmw:hardware --><\/p>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/higgs-audio-v3-stt-gguf\/\">higgs-audio-v3-stt-GGUF Speech-to-Text Model: 4GB+ VRAM, File List<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/soprano-1-1-80m-gguf-released\/\">Soprano-1.1-80M-GGUF Lightweight English TTS Model: File List<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/ggml-org-vibevoice-asr-gguf\/\">VibeVoice-ASR-GGUF Speech Recognition and Diarization Model: 8GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/qwen3-5-4b-gguf-released\/\">Qwen3.5-4B-GGUF Vision-Language Model: 4GB+ VRAM<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Find models by VRAM<\/strong> (This model runs from the 4GB tier) \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-8gb-en\/\">Other models that run on a 8GB GPU<\/a><\/li>\n<li><strong>Engines mentioned in this article<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-ggml-en\/\">ggml<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-llama-cpp-en\/\">llama.cpp<\/a><\/li>\n<li><strong>What Q2_K mean<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">Quantization and model-format glossary<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/huggingface.co\/ggml-org\/tinyLFM3-Omni-for-testing-gguf\">ggml-org\/tinyLFM3-Omni-for-testing-gguf<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>ggml-org released tinyLFM3-Omni-for-testing-gguf on Hugging Face to test llama.cpp multimodal and CI pipelines.<\/p>\n","protected":false},"author":1,"featured_media":15306,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[105],"tags":[161,1970,163,4708,2278,1073,4710,1547],"class_list":["post-15307","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-engines-and-tools","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-lfm3-omni-2b-en","tag-liquidai-en","tag-llama-cpp-en","tag-tinylfm3-omni-for-testing-gguf-en","tag-verified"],"lang":"en","translations":{"en":15307,"ja":15305},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/15307","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=15307"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/15307\/revisions"}],"predecessor-version":[{"id":18066,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/15307\/revisions\/18066"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/15306"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=15307"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=15307"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=15307"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}