{"id":13046,"date":"2026-11-17T11:01:57","date_gmt":"2026-11-17T02:01:57","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/?p=13046"},"modified":"2026-11-17T11:11:06","modified_gmt":"2026-11-17T02:11:06","slug":"ggml-org-flint-gguf","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/ggml-org-flint-gguf\/","title":{"rendered":"Flint-GGUF Coding MoE Language Model: 32GB+ VRAM"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/huggingface.co\/ggml-org\/Flint-GGUF\">ggml-org\/Flint-GGUF<\/a><\/td>\n<\/tr>\n<tr>\n<td>Family guide<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/model-recursivesi-flint-en\/\">flint guide (2 articles)<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-10-22<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>apache-2.0<\/td>\n<\/tr>\n<tr>\n<td>Formats<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-gguf-en\/\">GGUF<\/a><\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p><code>ggml-org\/Flint-GGUF<\/code>, a GGUF-converted repository for &#8220;Flint&#8221;\u2014a Mixture-of-Experts (MoE) language model for coding agents developed by Recursive\u2014has been released by ggml-org.<\/p>\n<p>The original model, Flint, is a large language model with a total of 39.8B parameters and approximately 2.5B active parameters per token. It was developed with the aim of handling practical software development agent tasks such as codebase investigation, bug fixing, new feature implementation, and test code creation. With the provision of the GGUF format compatible with the llama.cpp ecosystem, it has become easier to handle this model, which boasts a very long context length of 262K (262,144 tokens), in local PC and server environments.<\/p>\n<h2>Specifications<\/h2>\n<p>The specifications listed in the model cards of the original model and the GGUF version are as follows:<\/p>\n<ul>\n<li>Architecture: Decoder-only mixture-of-experts Transformer<\/li>\n<li>Total parameters: 39.8B<\/li>\n<li>Active parameters: Approximately 2.5B per token (including output head; active body alone is approximately 2.2B)<\/li>\n<li>Number of layers: 49 layers (2 dense layers + 47 MoE layers)<\/li>\n<li>Attention configuration: 35 sliding window layers, 14 full attention layers<\/li>\n<li>Sliding window length: 513 tokens (current token and 512 preceding tokens)<\/li>\n<li>Number of attention heads: 32 query heads, 4 key\/value heads, head dimension 128<\/li>\n<li>Attention normalization: Per-head QK RMSNorm and learned per-head attention sinks<\/li>\n<li>Positional encoding: RoPE adopted for sliding window layers, no positional encoding for full attention layers<\/li>\n<li>Expert configuration: 256 routed experts (selecting 6 experts per token) + 1 always-on shared expert<\/li>\n<li>Context length: 262,144 tokens (sum of prompt and generated output)<\/li>\n<li>Vocabulary size: 150,144 tokens<\/li>\n<li>Input\/Output format: Text input \/ Text output<\/li>\n<li>Knowledge cutoff: June 2026<\/li>\n<\/ul>\n<h2>Performance<\/h2>\n<p>Because the newly released repository is a GGUF quantized version, the performance benchmark figures are based on the model card of the original model, <code>RecursiveSI\/flint<\/code> (BF16 version). Please note that when operating the quantized version, accuracy may degrade slightly due to the quantization process.<\/p>\n<p>The evaluation results published by Recursive, the publisher of the original model, are as follows:<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th style=\"text-align: left;\">Benchmark<\/th>\n<th style=\"text-align: left;\">Score<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td style=\"text-align: left;\">Terminal-Bench 2.1<\/td>\n<td style=\"text-align: left;\">54.9%<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\">SWE-bench Pro v2<\/td>\n<td style=\"text-align: left;\">89.7%<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\">SWE-bench Verified<\/td>\n<td style=\"text-align: left;\">74.2%<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>From these evaluation results, it can be seen that Flint shows remarkably high performance in software development tasks and autonomous work in terminals. In particular, it records extremely high levels of 89.7% on SWE-bench Pro v2 and 74.2% on SWE-bench Verified, excelling in practical coding and problem-solving capabilities.<\/p>\n<p>Additionally, regarding the 54.9% result in Terminal-Bench 2.1, the publisher notes that cases where information was retrieved via web search to reach a solution were treated strictly as &#8220;unsuccessful.&#8221; Since aggregation conditions may not be identical to scores of other models published by the developers, caution is required when making comparisons.<\/p>\n<p>Note that these agent-related scores are calculated as an average pass@1 over at least 5 trials, and the inference generation settings specify a Temperature of 1.0, top-p of 0.95, and top-k of 20. The evaluation tools used are Terminus 2 for Terminal-Bench and mini-swe-agent for SWE-bench. Please be aware that these figures are measurement results published by the publisher itself and are not independent replication results by third parties.<\/p>\n<h2>Strengths and Use Cases<\/h2>\n<p>Flint is designed specifically for general agent tasks in software development. According to the model card of the original model <code>RecursiveSI\/flint<\/code>, the following development tasks are primarily cited as intended use cases:<\/p>\n<ul>\n<li>Structural investigation of existing codebases and understanding specifications<\/li>\n<li>Investigation and fixing of software bugs<\/li>\n<li>Implementation of new features and modules<\/li>\n<li>Creation of test code, including unit tests<\/li>\n<\/ul>\n<p>This model supports text input and output, and supports long reasoning processes and multi-turn tool calls. It is capable of calling tools in an OpenAI-compatible format, making it suitable for building agent environments that cooperate with external tools to autonomously execute tasks.<\/p>\n<p>In addition, equipped with a vast context window of 262,144 tokens, it is suited for use cases where large groups of source code files or lengthy dialogue histories are loaded at once to proceed with work.<\/p>\n<p>The generation parameters recommended by the publisher are a Temperature of 1.0, Top-p of 0.95, and Top-20 for Top-k. When tackling difficult problems, it is recommended to secure an output budget of at least 32,768 tokens, including the reasoning process. When conducting dialogues, it is required to use the chat template included with the model and keep the <code>reasoning_content<\/code> returned by the model in the conversation history as-is when sending.<\/p>\n<p>Note that the learning process was performed from scratch, ranging from pre-training and intermediate training to supervised fine-tuning (SFT) and reinforcement learning (RL), including text, code, instruction following, and agent tasks. On the other hand, the model card lists the following operational cautions:<\/p>\n<ul>\n<li>Although trained on safety-conscious datasets, it may generate misinformation, harmful content, code containing bugs, or inappropriate tool calls. The generated code must undergo testing, and human review is required before making significant changes.<\/li>\n<li>Applications that permit tool calls must strictly manage execution privileges and access boundaries. The syntactic rules for structured tool calls themselves do not prevent the model from selecting unsafe actions.<\/li>\n<li>Prolonged reasoning processes may increase response latency or reach the output token limit before the answer is completed.<\/li>\n<li>The context setting of 262,144 tokens indicates a capacity upper limit and does not guarantee uniform accuracy across all sequence lengths or tasks.<\/li>\n<\/ul>\n<p><!-- lmw:hardware --><\/p>\n<h2>Hardware Requirements<\/h2>\n<p><strong>Estimated requirements (calculated by Local Model Watch)<\/strong> \u2014 39.8B parameters (taken from the base model RecursiveSI\/flint)<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Your VRAM<\/th>\n<th>Quantization<\/th>\n<th>File size<\/th>\n<th>Est. memory needed<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>32GB (RTX 5090, etc.)<\/td>\n<td>Q4_K_M<\/td>\n<td>21.8GB<\/td>\n<td>26.1GB<\/td>\n<\/tr>\n<tr>\n<td>48GB (RTX 6000 Ada \/ A6000, etc.)<\/td>\n<td>Q8_0<\/td>\n<td>39.4GB<\/td>\n<td>47.3GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model&#8217;s authors. Compare with other models in our <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-en\/\">VRAM quick reference<\/a>. What the quantization names mean: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">glossary<\/a>.<\/em><\/p>\n<p><!-- \/lmw:hardware --><\/p>\n<p><!-- lmw:runnability --><\/p>\n<h2>Can You Run It Locally?<\/h2>\n<p><strong>Runs in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\n<p>It is distributed in GGUF, so no conversion is needed.<\/p>\n<p><strong>License \u2014 <code>apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\n<p><strong>Compression:<\/strong> the Q4_K_M build measures 4.71 bits per weight \u2014 about 29% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\n<p><em>Compiled by this site&#8217;s code from the published formats, converted builds we have found, and each engine&#8217;s own model registry. &#8220;Not found&#8221; means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher&#8217;s original terms before relying on them.<\/em><\/p>\n<p><!-- \/lmw:runnability --><\/p>\n<p><!-- lmw:peers --><\/p>\n<h2>Recent Models in the Same Size Class<\/h2>\n<p><em>Models with <\/em><em>15\u201340B<\/em><em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site&#8217;s estimates; licenses are as stated on the model cards.<\/em><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Model<\/th>\n<th>Parameters<\/th>\n<th>Smallest VRAM tier<\/th>\n<th>License<\/th>\n<th>Our article<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>LiquidAI\/LFM2.5-24B-A2B-Base<\/td>\n<td>24.0B<\/td>\n<td>80GB<\/td>\n<td>lfm1.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/liquidai-lfm2-5-24b-a2b-2\/\">LiquidAI Releases LFM2.5-24B-A2B for On-Device Agents<\/a> (2026-11-17)<\/td>\n<\/tr>\n<tr>\n<td>LiquidAI\/LFM2.5-24B-A2B-FP8<\/td>\n<td>24.0B<\/td>\n<td>32GB<\/td>\n<td>lfm1.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/liquidai-lfm2-5-24b-a2b-fp8-released\/\">LFM2.5-24B-A2B-FP8 Text Generation Model: 32GB+ VRAM<\/a> (2026-11-17)<\/td>\n<\/tr>\n<tr>\n<td>LiquidAI\/LFM2.5-24B-A2B-MLX-bf16<\/td>\n<td>24.0B<\/td>\n<td>80GB<\/td>\n<td>lfm1.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/liquidai-lfm2-5-24b-a2b-mlx-bf16-2\/\">LFM2.5-24B-A2B-MLX-bf16 Reasoning Model for Apple Silicon: 80GB+ VRAM<\/a> (2026-11-17)<\/td>\n<\/tr>\n<tr>\n<td>LiquidAI\/LFM2.5-24B-A2B-MLX-8bit<\/td>\n<td>24.0B<\/td>\n<td>32GB<\/td>\n<td>lfm1.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/liquidai-lfm2-5-24b-a2b-mlx-8bit\/\">LFM2.5-24B-A2B-MLX-8bit Text Generation Model: 32GB+ VRAM<\/a> (2026-11-17)<\/td>\n<\/tr>\n<tr>\n<td>LiquidAI\/LFM2.5-24B-A2B<\/td>\n<td>24.0B<\/td>\n<td>12GB<\/td>\n<td>lfm1.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/liquidai-lfm25-24b-a2b\/\">LFM2.5-24B-A2B Agent-Optimized MoE Reasoning Model: 12GB+ VRAM<\/a> (2026-11-17)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><!-- \/lmw:peers --><\/p>\n<h2>How to Get It<\/h2>\n<p>The GGUF version released this time is distributed in the <code>ggml-org\/Flint-GGUF<\/code> repository on Hugging Face. It is published under the Apache 2.0 license and can be downloaded directly without requiring prior agreement to terms of use (gated application).<\/p>\n<p>Distributed variations include:<\/p>\n<ul>\n<li><code>BF16<\/code><\/li>\n<li><code>Q8_0<\/code><\/li>\n<li><code>Q4_K_M<\/code><\/li>\n<\/ul>\n<p>In <code>Q4_K_M<\/code>, smaller non-expert tensors such as embedding layers, attention, and shared experts are specified to be maintained at <code>Q8_0<\/code> precision. Furthermore, this model is text-only and does not include additional components such as a multimodal projector (mmproj), MTP, or DFlash.<\/p>\n<p>Running this model requires llama.cpp with added architecture support for Flint (a build corresponding to Pull Request #30928). ggml-org&#8217;s automated conversion script was used for model conversion.<\/p>\n<p>When using the server functionality of llama.cpp, you can fetch the model and launch a local server by running the following command from the terminal:<\/p>\n<pre><code class=\"language-bash\">llama serve -hf ggml-org\/Flint-GGUF\n<\/code><\/pre>\n<p>Additionally, for users who wish to use it in a GUI environment, execution on <a href=\"https:\/\/llama.app\">Llama app<\/a> is also guided.<\/p>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/flint-39-8b-moe-coding-agent\/\">flint MoE Language Model for Coding Agents: 16GB+ VRAM, GGUF Builds<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/nvidia-nemotron-3-5-bolt-e4b-gguf-2\/\">NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF Text Generation Model: 12GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/qwen3-coder-30b-a3b-instruct-gguf-2\/\">Qwen3-Coder-30B-A3B-Instruct-GGUF Text Generation Model: 24GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/vllm-d3-gguf-models-released\/\">vllm-d3 GGUF Models Released: Decision Models for Local Run<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Find models by VRAM<\/strong> (This model needs at least 32GB) \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-en\/\">VRAM quick reference<\/a><\/li>\n<li><strong>Explore the same model family<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/model-recursivesi-flint-en\/\">flint family overview (2 articles, 5 converted builds)<\/a><\/li>\n<li><strong>Engines that run this model<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-llama-cpp-en\/\">llama.cpp<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-ollama-en\/\">Ollama<\/a><\/li>\n<li><strong>What Q4_K_M, Q8_0 mean and where to get this model<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-gguf-en\/\">GGUF format guide and models<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">Quantization and model-format glossary<\/a><\/li>\n<li><strong>Other models for the same task<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/models-by-task-en\/#task-text\">Other text generation models<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/huggingface.co\/ggml-org\/Flint-GGUF\">ggml-org\/Flint-GGUF (Hugging Face)<\/a><\/li>\n<li><a href=\"https:\/\/huggingface.co\/RecursiveSI\/flint\">RecursiveSI\/flint (Hugging Face)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/30928\">llama.cpp Pull Request #30928<\/a><\/li>\n<li><a href=\"https:\/\/llama.app\">Llama app<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>The GGUF version of Flint, a 39.8B parameter coding agent MoE language model, has been released by ggml-org for local execution.<\/p>\n","protected":false},"author":1,"featured_media":13045,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[3976,161,163,1073,165,3978,1547,1555],"class_list":["post-13046","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-flint-en","tag-ggml-en","tag-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-recursivesi-en","tag-verified","tag--en"],"lang":"en","translations":{"en":13046,"ja":13044},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/13046","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=13046"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/13046\/revisions"}],"predecessor-version":[{"id":17907,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/13046\/revisions\/17907"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/13045"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=13046"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=13046"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=13046"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}