{"id":13050,"date":"2026-11-17T11:01:58","date_gmt":"2026-11-17T02:01:58","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/?p=13050"},"modified":"2026-11-17T11:11:08","modified_gmt":"2026-11-17T02:11:08","slug":"qwen3-coder-30b-a3b-instruct-gguf-2","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/qwen3-coder-30b-a3b-instruct-gguf-2\/","title":{"rendered":"Qwen3-Coder-30B-A3B-Instruct-GGUF Text Generation Model: 24GB+ VRAM"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/huggingface.co\/ggml-org\/Qwen3-Coder-30B-A3B-Instruct-GGUF\">ggml-org\/Qwen3-Coder-30B-A3B-Instruct-GGUF<\/a><\/td>\n<\/tr>\n<tr>\n<td>Publisher guide<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/publisher-alibaba-en\/\">Alibaba (Qwen): models and licenses<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-10-22<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>apache-2.0<\/td>\n<\/tr>\n<tr>\n<td>Formats<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-gguf-en\/\">GGUF<\/a><\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p><code>ggml-org\/Qwen3-Coder-30B-A3B-Instruct-GGUF<\/code>, a GGUF format quantized model based on the Qwen team&#8217;s coding-focused model &#8220;Qwen3-Coder-30B-A3B-Instruct&#8221;, has been released by ggml-org. The Qwen team is known for developing numerous open-weight large language models.<\/p>\n<p>The original model, Qwen3-Coder-30B-A3B-Instruct, adopts a highly efficient Mixture of Experts (MoE) architecture that limits the number of active parameters during inference to 3.3B out of a total parameter count of 30.5B. It is designed with agentic coding assistance, tool calling, and long-context processing in mind, aiming to achieve both high execution performance and resource efficiency.<\/p>\n<p>The newly released GGUF version was automatically converted using conversion tools provided by ggml-org. It is provided as a model file that enables quick and easy inference in local inference environments such as llama.cpp.<\/p>\n<h2>Specifications<\/h2>\n<p>Specifications listed in the publisher&#8217;s and original model&#8217;s model cards are as follows:<\/p>\n<ul>\n<li>Parameters: Total parameters 30.5B (Active parameters 3.3B)<\/li>\n<li>Architecture: Mixture of Experts (Qwen3MoeForCausalLM \/ qwen3_moe)<\/li>\n<li>Layers: 48<\/li>\n<li>Attention mechanism: Grouped-Query Attention (GQA, 32 Query heads \/ 4 Key-Value heads)<\/li>\n<li>Experts: Total experts 128 (8 active experts per token)<\/li>\n<li>Context length: Native 262,144 tokens (highlighted as 256K tokens in the original model, extendable up to 1M tokens using Yarn)<\/li>\n<li>Training stage: Pretraining and post-training<\/li>\n<li>Operating mode: Supports non-thinking mode only (does not generate thinking blocks during output, and explicit <code>enable_thinking=False<\/code> specification during inference is not required)<\/li>\n<li>Recommended output length: 65,536 tokens<\/li>\n<\/ul>\n<p>Additionally, the following quantization types are available in this GGUF repository:<\/p>\n<ul>\n<li>BF16<\/li>\n<li>Q8_0<\/li>\n<li>Q4_K_M<\/li>\n<\/ul>\n<p>As part of the quantization implementation specifications, respective base quantization types are applied to the routed expert layers in Q8_0 and Q4_K_M. Furthermore, in Q4_K_M, a hybrid configuration is adopted where smaller non-expert tensors such as embeddings and attention layers are retained at Q8_0 precision, designed to suppress degradation in accuracy caused by quantization.<\/p>\n<h2>Performance<\/h2>\n<p>Because this model is a GGUF converted version by ggml-org, performance evaluations conform to the original model, Qwen team&#8217;s &#8220;Qwen\/Qwen3-Coder-30B-A3B-Instruct&#8221;.<\/p>\n<p>While detailed benchmark score tables are not posted in the original model&#8217;s model card, qualitative performance evaluations by the publisher are explicitly stated. According to the publisher&#8217;s description, the model demonstrates remarkably superior performance among open models in &#8220;Agentic Coding&#8221;, &#8220;Agentic Browser-Use&#8221;, and general foundational coding tasks.<\/p>\n<p>In particular, tool-calling capabilities have been enhanced, and dedicated function-calling formats designed for major coding assistance platforms such as Qwen Code and CLINE are incorporated. In addition, because it features ultra-long context processing capabilities of 262,144 tokens natively and up to 1M tokens via extension, it is described as having high practical utility in understanding and editing large-scale code structures across entire repositories.<\/p>\n<p>Note that these performance evaluations are qualitative remarks published by the model&#8217;s creators themselves and are not detailed comparative figures from third-party institutions or standard benchmarks. Furthermore, it should be noted that during the quantization process from the original BF16 to Q8_0 or Q4_K_M, slight degradations in output quality and behavior may occur compared to the original model due to reduced calculation precision.<\/p>\n<h2>Strengths and Use Cases<\/h2>\n<p>This model is designed specifically for automating coding processes and analyzing and editing large-scale development projects. It inherits the excellent features of its quantization source, &#8220;Qwen3-Coder-30B-A3B-Instruct&#8221;.<\/p>\n<p>Main strengths and expected use cases include:<\/p>\n<ul>\n<li><strong>Agentic Coding<\/strong>: Compatible with major automated development tools and agent environments such as Qwen Code and CLINE. Because dedicated function-calling formats are designed, it is easy to integrate into systems that accurately call defined tools to automate numerical calculations or external API operations. It is also suitable for defining and calling tools via OpenAI API-compatible endpoints (such as <code>http:\/\/localhost:8000\/v1<\/code>).<\/li>\n<li><strong>Repository Understanding Leveraging Ultra-Long Contexts<\/strong>: Supports long-context processing up to 262,144 tokens natively, and up to 1M tokens using Yarn. This allows you to go beyond single-file processing and feed entire sets of source code or vast documentation from a whole repository into the context at once, enabling project-wide code generation, bug analysis, and refactoring.<\/li>\n<li><strong>Agentic Browser-Use and Various Development Tasks<\/strong>: Handles a diverse range of foundational programming tasks that arise in software development, including Agentic Browser-Use.<\/li>\n<\/ul>\n<p>Note that this model supports only &#8220;non-thinking mode&#8221;, which does not involve thinking processes in its outputs. Therefore, there is no need to explicitly specify <code>enable_thinking=False<\/code> during inference execution, and answers can be obtained straightforwardly.<\/p>\n<p>As sampling parameters to best draw out the model&#8217;s performance, the developers recommend the following settings:<\/p>\n<ul>\n<li><code>temperature<\/code>: 0.7<\/li>\n<li><code>top_p<\/code>: 0.8<\/li>\n<li><code>top_k<\/code>: 20<\/li>\n<li><code>repetition_penalty<\/code>: 1.05<\/li>\n<\/ul>\n<p>Additionally, to ensure a sufficient response length, it is recommended to set the maximum output length (<code>max_new_tokens<\/code>) to 65,536 tokens for general queries.<\/p>\n<p><!-- lmw:hardware --><\/p>\n<h2>Hardware Requirements<\/h2>\n<p><strong>Estimated requirements (calculated by Local Model Watch)<\/strong> \u2014 30.5B parameters (taken from the base model Qwen\/Qwen3-Coder-30B-A3B-Instruct)<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Your VRAM<\/th>\n<th>Quantization<\/th>\n<th>File size<\/th>\n<th>Est. memory needed<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>24GB (RTX 4090 \/ 3090, etc.)<\/td>\n<td>Q4_K_M<\/td>\n<td>16.8GB<\/td>\n<td>20.1GB<\/td>\n<\/tr>\n<tr>\n<td>48GB (RTX 6000 Ada \/ A6000, etc.)<\/td>\n<td>Q8_0<\/td>\n<td>30.3GB<\/td>\n<td>36.3GB<\/td>\n<\/tr>\n<tr>\n<td>80GB class (A100 \/ H100)<\/td>\n<td>BF16<\/td>\n<td>56.9GB<\/td>\n<td>68.3GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model&#8217;s authors. Compare with other models in our <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-en\/\">VRAM quick reference<\/a>. What the quantization names mean: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">glossary<\/a>.<\/em><\/p>\n<p><!-- \/lmw:hardware --><\/p>\n<p><!-- lmw:runnability --><\/p>\n<h2>Can You Run It Locally?<\/h2>\n<p><strong>Runs in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\n<p>It is distributed in GGUF, so no conversion is needed.<\/p>\n<p><strong>License \u2014 <code>apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\n<p><strong>Compression:<\/strong> the Q4_K_M build measures 4.73 bits per weight \u2014 about 30% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\n<p><em>Compiled by this site&#8217;s code from the published formats, converted builds we have found, and each engine&#8217;s own model registry. &#8220;Not found&#8221; means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher&#8217;s original terms before relying on them.<\/em><\/p>\n<p><!-- \/lmw:runnability --><\/p>\n<p><!-- lmw:peers --><\/p>\n<h2>Recent Models in the Same Size Class<\/h2>\n<p><em>Models with <\/em><em>15\u201340B<\/em><em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site&#8217;s estimates; licenses are as stated on the model cards.<\/em><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Model<\/th>\n<th>Parameters<\/th>\n<th>Smallest VRAM tier<\/th>\n<th>License<\/th>\n<th>Our article<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>LiquidAI\/LFM2.5-24B-A2B-Base<\/td>\n<td>24.0B<\/td>\n<td>80GB<\/td>\n<td>lfm1.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/liquidai-lfm2-5-24b-a2b-2\/\">LiquidAI Releases LFM2.5-24B-A2B for On-Device Agents<\/a> (2026-11-17)<\/td>\n<\/tr>\n<tr>\n<td>LiquidAI\/LFM2.5-24B-A2B-FP8<\/td>\n<td>24.0B<\/td>\n<td>32GB<\/td>\n<td>lfm1.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/liquidai-lfm2-5-24b-a2b-fp8-released\/\">LFM2.5-24B-A2B-FP8 Text Generation Model: 32GB+ VRAM<\/a> (2026-11-17)<\/td>\n<\/tr>\n<tr>\n<td>LiquidAI\/LFM2.5-24B-A2B-MLX-bf16<\/td>\n<td>24.0B<\/td>\n<td>80GB<\/td>\n<td>lfm1.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/liquidai-lfm2-5-24b-a2b-mlx-bf16-2\/\">LFM2.5-24B-A2B-MLX-bf16 Reasoning Model for Apple Silicon: 80GB+ VRAM<\/a> (2026-11-17)<\/td>\n<\/tr>\n<tr>\n<td>LiquidAI\/LFM2.5-24B-A2B-MLX-8bit<\/td>\n<td>24.0B<\/td>\n<td>32GB<\/td>\n<td>lfm1.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/liquidai-lfm2-5-24b-a2b-mlx-8bit\/\">LFM2.5-24B-A2B-MLX-8bit Text Generation Model: 32GB+ VRAM<\/a> (2026-11-17)<\/td>\n<\/tr>\n<tr>\n<td>LiquidAI\/LFM2.5-24B-A2B<\/td>\n<td>24.0B<\/td>\n<td>12GB<\/td>\n<td>lfm1.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/liquidai-lfm25-24b-a2b\/\">LFM2.5-24B-A2B Agent-Optimized MoE Reasoning Model: 12GB+ VRAM<\/a> (2026-11-17)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><!-- \/lmw:peers --><\/p>\n<h2>How to Get It<\/h2>\n<p>This model is available from the Hugging Face repository <code>ggml-org\/Qwen3-Coder-30B-A3B-Instruct-GGUF<\/code>. It is distributed under the Apache License 2.0 (apache-2.0), allowing users to freely download and use it without undergoing individual agreement procedures for license terms.<\/p>\n<p>It is distributed in GGUF format with the following quantization precisions provided:<\/p>\n<ul>\n<li><code>BF16<\/code><\/li>\n<li><code>Q8_0<\/code><\/li>\n<li><code>Q4_K_M<\/code><\/li>\n<\/ul>\n<p>Among these, <code>Q4_K_M<\/code> and <code>Q8_0<\/code> apply respective quantization to the routed expert layers, while <code>Q4_K_M<\/code> maintains non-expert layers such as embeddings and attention layers at <code>Q8_0<\/code> precision. This is designed to allow efficient operation in local environments while suppressing accuracy degradation.<\/p>\n<p>This model can be used with major inference engines and applications supporting GGUF, such as llama.cpp, Ollama, LM Studio, MLX-LM, and KTransformers.<\/p>\n<p>For example, when using the llama.app environment, you can launch a server by directly loading the model from Hugging Face by running the following command:<\/p>\n<pre><code class=\"language-bash\">llama serve -hf ggml-org\/Qwen3-Coder-30B-A3B-Instruct-GGUF\n<\/code><\/pre>\n<p>Note that if you perform direct inference using the original model&#8217;s Python code or the <code>transformers<\/code> library, older versions like <code>transformers&lt;4.51.0<\/code> will raise a <code>KeyError: 'qwen3_moe'<\/code>, so using the latest <code>transformers<\/code> library is recommended. Additionally, if out-of-memory errors occur during local inference, reducing the context length to a shorter value such as <code>32,768<\/code> tokens is guided as a workaround.<\/p>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/vllm-d3-gguf-models-released\/\">vllm-d3 GGUF Models Released: Decision Models for Local Run<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/02\/ggml-org-openjev-gguf-released-2\/\">OpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/muse-glimmer-30b-gguf-2\/\">Muse-Glimmer-30B-GGUF 30B Multimodal Local Agent Model: 24GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/04\/glm-5-3-flash-gguf-released\/\">GLM-5.3-Flash-GGUF Vision-Language Model: ~150GB Memory<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Find models by VRAM<\/strong> (This model runs from the 24GB tier) \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-24gb-en\/\">Other models that run on a 24GB GPU<\/a><\/li>\n<li><strong>Engines that run this model<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-llama-cpp-en\/\">llama.cpp<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-ollama-en\/\">Ollama<\/a><\/li>\n<li><strong>What Q4_K_M, Q8_0 mean and where to get this model<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-gguf-en\/\">GGUF format guide and models<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">Quantization and model-format glossary<\/a><\/li>\n<li><strong>Learn about the publisher<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/publisher-alibaba-en\/\">Alibaba (Qwen): models, licenses and articles<\/a><\/li>\n<li><strong>Other models for the same task<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/models-by-task-en\/#task-text\">Other text generation models<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/huggingface.co\/ggml-org\/Qwen3-Coder-30B-A3B-Instruct-GGUF\">https:\/\/huggingface.co\/ggml-org\/Qwen3-Coder-30B-A3B-Instruct-GGUF<\/a><\/li>\n<li><a href=\"https:\/\/huggingface.co\/Qwen\/Qwen3-Coder-30B-A3B-Instruct\">https:\/\/huggingface.co\/Qwen\/Qwen3-Coder-30B-A3B-Instruct<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Explore the GGUF quantization of Qwen3-Coder-30B-A3B-Instruct, an efficient MoE coding model featuring agentic capabilities and long context.<\/p>\n","protected":false},"author":1,"featured_media":13049,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1073,165,3984,1547,1555],"class_list":["post-13050","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-qwen3-coder-en","tag-verified","tag--en"],"lang":"en","translations":{"en":13050,"ja":13048},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/13050","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=13050"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/13050\/revisions"}],"predecessor-version":[{"id":17909,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/13050\/revisions\/17909"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/13049"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=13050"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=13050"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=13050"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}