Qwen3-Coder-30B-A3B-Instruct-GGUF Text Generation Model: 24GB+ VRAM

Qwen3-Coder-30B-A3B-Instruct-GGUF Text Generation Model: 24GB+ VRAM

At a Glance

Item Value
Repository ggml-org/Qwen3-Coder-30B-A3B-Instruct-GGUF
Publisher guide Alibaba (Qwen): models and licenses
Published 2026-10-22
License apache-2.0
Formats GGUF
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

ggml-org/Qwen3-Coder-30B-A3B-Instruct-GGUF, a GGUF format quantized model based on the Qwen team’s coding-focused model “Qwen3-Coder-30B-A3B-Instruct", has been released by ggml-org. The Qwen team is known for developing numerous open-weight large language models.

The original model, Qwen3-Coder-30B-A3B-Instruct, adopts a highly efficient Mixture of Experts (MoE) architecture that limits the number of active parameters during inference to 3.3B out of a total parameter count of 30.5B. It is designed with agentic coding assistance, tool calling, and long-context processing in mind, aiming to achieve both high execution performance and resource efficiency.

The newly released GGUF version was automatically converted using conversion tools provided by ggml-org. It is provided as a model file that enables quick and easy inference in local inference environments such as llama.cpp.

Specifications

Specifications listed in the publisher’s and original model’s model cards are as follows:

  • Parameters: Total parameters 30.5B (Active parameters 3.3B)
  • Architecture: Mixture of Experts (Qwen3MoeForCausalLM / qwen3_moe)
  • Layers: 48
  • Attention mechanism: Grouped-Query Attention (GQA, 32 Query heads / 4 Key-Value heads)
  • Experts: Total experts 128 (8 active experts per token)
  • Context length: Native 262,144 tokens (highlighted as 256K tokens in the original model, extendable up to 1M tokens using Yarn)
  • Training stage: Pretraining and post-training
  • Operating mode: Supports non-thinking mode only (does not generate thinking blocks during output, and explicit enable_thinking=False specification during inference is not required)
  • Recommended output length: 65,536 tokens

Additionally, the following quantization types are available in this GGUF repository:

  • BF16
  • Q8_0
  • Q4_K_M

As part of the quantization implementation specifications, respective base quantization types are applied to the routed expert layers in Q8_0 and Q4_K_M. Furthermore, in Q4_K_M, a hybrid configuration is adopted where smaller non-expert tensors such as embeddings and attention layers are retained at Q8_0 precision, designed to suppress degradation in accuracy caused by quantization.

Performance

Because this model is a GGUF converted version by ggml-org, performance evaluations conform to the original model, Qwen team’s “Qwen/Qwen3-Coder-30B-A3B-Instruct".

While detailed benchmark score tables are not posted in the original model’s model card, qualitative performance evaluations by the publisher are explicitly stated. According to the publisher’s description, the model demonstrates remarkably superior performance among open models in “Agentic Coding", “Agentic Browser-Use", and general foundational coding tasks.

In particular, tool-calling capabilities have been enhanced, and dedicated function-calling formats designed for major coding assistance platforms such as Qwen Code and CLINE are incorporated. In addition, because it features ultra-long context processing capabilities of 262,144 tokens natively and up to 1M tokens via extension, it is described as having high practical utility in understanding and editing large-scale code structures across entire repositories.

Note that these performance evaluations are qualitative remarks published by the model’s creators themselves and are not detailed comparative figures from third-party institutions or standard benchmarks. Furthermore, it should be noted that during the quantization process from the original BF16 to Q8_0 or Q4_K_M, slight degradations in output quality and behavior may occur compared to the original model due to reduced calculation precision.

Strengths and Use Cases

This model is designed specifically for automating coding processes and analyzing and editing large-scale development projects. It inherits the excellent features of its quantization source, “Qwen3-Coder-30B-A3B-Instruct".

Main strengths and expected use cases include:

  • Agentic Coding: Compatible with major automated development tools and agent environments such as Qwen Code and CLINE. Because dedicated function-calling formats are designed, it is easy to integrate into systems that accurately call defined tools to automate numerical calculations or external API operations. It is also suitable for defining and calling tools via OpenAI API-compatible endpoints (such as http://localhost:8000/v1).
  • Repository Understanding Leveraging Ultra-Long Contexts: Supports long-context processing up to 262,144 tokens natively, and up to 1M tokens using Yarn. This allows you to go beyond single-file processing and feed entire sets of source code or vast documentation from a whole repository into the context at once, enabling project-wide code generation, bug analysis, and refactoring.
  • Agentic Browser-Use and Various Development Tasks: Handles a diverse range of foundational programming tasks that arise in software development, including Agentic Browser-Use.

Note that this model supports only “non-thinking mode", which does not involve thinking processes in its outputs. Therefore, there is no need to explicitly specify enable_thinking=False during inference execution, and answers can be obtained straightforwardly.

As sampling parameters to best draw out the model’s performance, the developers recommend the following settings:

  • temperature: 0.7
  • top_p: 0.8
  • top_k: 20
  • repetition_penalty: 1.05

Additionally, to ensure a sufficient response length, it is recommended to set the maximum output length (max_new_tokens) to 65,536 tokens for general queries.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch) — 30.5B parameters (taken from the base model Qwen/Qwen3-Coder-30B-A3B-Instruct)

Your VRAM Quantization File size Est. memory needed
24GB (RTX 4090 / 3090, etc.) Q4_K_M 16.8GB 20.1GB
48GB (RTX 6000 Ada / A6000, etc.) Q8_0 30.3GB 36.3GB
80GB class (A100 / H100) BF16 56.9GB 68.3GB

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

Runs in Ollama, LM Studio and llama.cpp as-is.

It is distributed in GGUF, so no conversion is needed.

License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.

Compression: the Q4_K_M build measures 4.73 bits per weight — about 30% the size of the original 16-bit weights, calculated by this site from the actual file sizes.

Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Recent Models in the Same Size Class

Models with 15–40B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.

→ Scroll horizontally to see all columns

Model Parameters Smallest VRAM tier License Our article
LiquidAI/LFM2.5-24B-A2B-Base 24.0B 80GB lfm1.0 LiquidAI Releases LFM2.5-24B-A2B for On-Device Agents (2026-11-17)
LiquidAI/LFM2.5-24B-A2B-FP8 24.0B 32GB lfm1.0 LFM2.5-24B-A2B-FP8 Text Generation Model: 32GB+ VRAM (2026-11-17)
LiquidAI/LFM2.5-24B-A2B-MLX-bf16 24.0B 80GB lfm1.0 LFM2.5-24B-A2B-MLX-bf16 Reasoning Model for Apple Silicon: 80GB+ VRAM (2026-11-17)
LiquidAI/LFM2.5-24B-A2B-MLX-8bit 24.0B 32GB lfm1.0 LFM2.5-24B-A2B-MLX-8bit Text Generation Model: 32GB+ VRAM (2026-11-17)
LiquidAI/LFM2.5-24B-A2B 24.0B 12GB lfm1.0 LFM2.5-24B-A2B Agent-Optimized MoE Reasoning Model: 12GB+ VRAM (2026-11-17)

How to Get It

This model is available from the Hugging Face repository ggml-org/Qwen3-Coder-30B-A3B-Instruct-GGUF. It is distributed under the Apache License 2.0 (apache-2.0), allowing users to freely download and use it without undergoing individual agreement procedures for license terms.

It is distributed in GGUF format with the following quantization precisions provided:

  • BF16
  • Q8_0
  • Q4_K_M

Among these, Q4_K_M and Q8_0 apply respective quantization to the routed expert layers, while Q4_K_M maintains non-expert layers such as embeddings and attention layers at Q8_0 precision. This is designed to allow efficient operation in local environments while suppressing accuracy degradation.

This model can be used with major inference engines and applications supporting GGUF, such as llama.cpp, Ollama, LM Studio, MLX-LM, and KTransformers.

For example, when using the llama.app environment, you can launch a server by directly loading the model from Hugging Face by running the following command:

llama serve -hf ggml-org/Qwen3-Coder-30B-A3B-Instruct-GGUF

Note that if you perform direct inference using the original model’s Python code or the transformers library, older versions like transformers<4.51.0 will raise a KeyError: 'qwen3_moe', so using the latest transformers library is recommended. Additionally, if out-of-memory errors occur during local inference, reducing the context length to a shorter value such as 32,768 tokens is guided as a workaround.

Related Articles

What to Read Next

Sources