Ling-3.1-flash Hybrid Reasoning MoE Model: ~641GB Memory, GGUF Builds

At a Glance
| Item | Value |
|---|---|
| Repository | inclusionAI/Ling-3.1-flash |
| Family guide | Ling-3.1 guide (3 articles) |
| Publisher guide | inclusionAI (Ant Group): models and licenses |
| Published | 2026-10-13 |
| License | mit |
| Formats | safetensors |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
inclusionAI has released the hybrid reasoning model “Ling-3.1-flash". It is positioned as an upgraded version based on the previous generation Ling-3.1-flash, with improvements to its architecture and performance on practical tasks. It is a MoE model with 560B total parameters and 25B active parameters, supporting a long context length of 1M tokens, and is designed with real-world tasks such as agentic tasks, software development, and office work in mind.
Specifications
The configuration listed in the model card is as follows:
- Total Parameters: 560B
- Active Parameters (per token): 25B
- Architecture: Hybrid-linear MoE
- Context Length: 1M tokens
- Transformer Layer Configuration: KDA 63 layers + Gated MLA 9 layers (7:1 ratio)
- Dense Layers: 4
- Routed Experts: 512
- Shared Experts: 1
- Activated Experts (per token): 8
- Attention Heads: 64
- Hidden Size: 4096
- Expert Intermediate Size: 1280
- Dense Intermediate Size: 10240
- Vocabulary Size: 157184
Performance
The model card does not include a comparative table showing specific scores, and evaluation results are mentioned only in text. According to the publishers, Ling-3.1-flash shows strong performance on coding and agent benchmarks such as Terminal-Bench 4.0, SWE-bench Pro, FrontierSWE, GDPVal-AA V2.1, Automationbench, MCP-Atlas, and SkillsBench.
Among those whose contents can be confirmed via benchmark_glossary, Terminal-Bench is a metric measuring whether commands can be actually executed in a terminal to complete given tasks using Pass@1(%), and SWE-bench is a metric measuring whether actual GitHub issues can be resolved by editing repositories using Resolved (percentage of solved problems, %). Both are metrics where higher is better, and the publishers state that it performs “strongly" on benchmark groups including these, though specific numbers cannot be confirmed from the materials.
Additionally, it is reported that good user experiences can be obtained when used on actual agentic frameworks such as Claude Code, Kilo Code, Qwen Code, Hermes Agent, and OpenClaw, and it is explained that it exhibits strong performance on practical tasks in fields such as STEM, medicine, and finance beyond coding agent use cases. Furthermore, for long-term tasks involving long durations and hundreds of tool calls, a proprietary harness called “LingCode" based on the open-source coding agent “Pi" is introduced, though this is an ingenuity on the execution environment side rather than the model itself, and open-sourcing is reportedly under consideration for the future. Note that these are all evaluations and explanations by the publishers themselves and are not third-party verification results.
Strengths and Use Cases
Ling-3.1-flash is designed with a strong focus on agent use cases that autonomously handle complex tasks in the real world. The model card lists general-purpose agents, search, office productivity enhancement, and software engineering as its main applications.
Specifically, it is suited for tasks involving tool execution and command operations using a terminal, and task automation through agent frameworks such as Claude Code and OpenClaw. Furthermore, performance improvements are mentioned in domains requiring advanced specialized knowledge such as general STEM fields, medicine, finance, and materials science. Because it features a wide context window reaching 1M tokens, it is suited for workflows where maintaining context is essential, such as reading lengthy documents or long-term coding tasks repeating numerous tool calls.
In addition, an option to enable the thinking process in the chat template (enable_thinking) is provided, and utilization in logical tasks and problem-solving solved step-by-step with reasoning in between is also envisioned.
Hardware Requirements
Estimated requirements (calculated by Local Model Watch) — 572.1B parameters
| Your VRAM | Quantization | File size | Est. memory needed |
|---|---|---|---|
| More than 641GB of VRAM (multi-GPU or CPU offload required) | FP8 | 534.4GB | 641.3GB |
Inference engine support (architecture name matched against each project’s own model registry in its source code, checked 2026-11-17): llama.cpp: registered, vLLM: registered, MLX (mlx-lm): registered.
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.
Can You Run It Locally?
Runs in Ollama, LM Studio and llama.cpp via a converted build.
The publisher ships safetensors, but bartowski/Ling-3.1-flash-GGUF provides a GGUF build you can use.
License — mit (Commercial use allowed): Permits commercial use, modification and redistribution, provided the copyright notice and license text are retained.
Compression: the FP8 build measures 8.02 bits per weight — about 50% the size of the original 16-bit weights, calculated by this site from the actual file sizes.
Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.
Recent Models in the Same Size Class
Models with over 40B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.
→ Scroll horizontally to see all columns
| Model | Parameters | Smallest VRAM tier | License | Our article |
|---|---|---|---|---|
| inclusionAI/Ling-3.1-flash-1019 | 572.1B | — | mit | Ling-3.1-flash-1019 Text Generation Model: ~641GB Memory, GGUF Builds (2026-11-17) |
| inclusionAI/Ling-3.1-flash-base | 572.1B | — | mit | Ling-3.1-flash-base Released: 560B Hybrid Linear MoE Model (2026-11-17) |
| inclusionAI/Ling-3.0-flash-Sante-fp8 | 127.5B | — | mit | Ling-3.0-flash-Sante-fp8 Text Generation Model: ~144GB Memory (2026-11-17) |
| deepseek-ai/DeepSeek-V4-Pro-0813 | 1650.5B | — | mit | DeepSeek-V4-Pro-0813 Text Generation Model: ~998GB Memory, GGUF Builds (2026-09-26) |
| nvidia/DeepSeek-V4-Pro-0813-nvfp4-DSpark | 1650.5B | — | mit | DeepSeek-V4-Pro-0813-nvfp4-DSpark: ~1005GB Memory (2026-09-10) |
How to Get It
The weights of Ling-3.1-flash are published on Hugging Face in FP8 safetensors format, and the license is the MIT License. Gate application (individual consent to terms of use) is not required for use, and it can be downloaded and used directly.
As execution engines, setup procedures are introduced for two types: SGLang, which is guided as the recommended environment, and vLLM.
Running with SGLang (Recommended)
An official Docker image containing support for Ling-3.1-flash, MTP (NEXTN) speculative decoding, and decode optimizations for Hopper is provided.
Pulling the image:
docker pull lmsysorg/sglang:dev-ling-3.1-flash
Starting the server (e.g., specifying tensor parallelism degree 8 and expert parallelism degree 8 in an 8x H200 environment):
docker run --gpus all --network host --ipc=host --rm \
lmsysorg/sglang:dev-ling-3.1-flash \
python3 -m sglang.launch_server \
--model-path inclusionAI/Ling-3.1-flash \
--tp 8 --ep 8 \
--speculative-algorithm NEXTN \
--host 0.0.0.0 --port 30000
Note that for a configuration using 4 GB300 units, specify --tp 4 --ep 4. In addition, decoding speedup through Hopper optimizations can be enabled by supplying the environment variables SGLANG_ENABLE_FP8_BLOCK_GEMV=1 and SGLANG_ENABLE_FUSED_KDA_OUTPUT_QUANT=1.
Example inference request from a client:
curl -s http://localhost:30000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model": "Ling-3.1-flash",
"messages": [{"role": "user", "content": "What is the capital of France?"}],
"chat_template_kwargs": {"enable_thinking": true},
"stream": true,
"temperature": 1.0,
"top_k": 20,
"top_p": 0.95}'
Running with vLLM
When using vLLM, building a virtual environment using uv and building from the vLLM repository are guided.
Environment setup procedure:
pip install uv
uv venv ~/my_ling_env
source ~/my_ling_env/bin/activate
git clone -b vllm https://github.com/vllm-project/vllm.git
cd vllm
VLLM_USE_PRECOMPILED=1 uv pip install --editable. --torch-backend=auto
Starting the server:
vllm serve "$MODEL_PATH" \
--port "$PORT" \
--trust-remote-code \
--served-model-name auto \
--tensor-parallel-size 8 \
--gpu-memory-utilization 0.85 \
--enable-prefix-caching \
--enable-expert-parallel \
--mamba-cache-mode align \
--enable-auto-tool-choice \
--tool-call-parser ling3 \
--reasoning-parser ling3 \
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'
Example inference request from a client:
curl -s http://${MASTER_IP}:${PORT}/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model": "auto",
"messages": [{"role": "user", "content": "What is the capital of France?"}],
"chat_template_kwargs": {"enable_thinking": true},
"stream": true,
"temperature": 1.0,
"top_k": 20,
"top_p": 0.95}'
As sampling parameters, a combination of temperature=1.0, top_p=0.95, and top_k=20 is recommended, and enabling enable_thinking is recommended to fully bring out performance.
Quantized and Converted Variants
→ Scroll horizontally to see all columns
| Added | Publisher | Format | Repository | Smallest VRAM tier (build, est. memory) |
|---|---|---|---|---|
| 2026-11-17 | inclusionAI | FP8 | inclusionAI/Ling-3.1-flash-1019 | FP8 641.3GB (does not fit a single consumer GPU) |
| 2026-11-17 | bartowski | GGUF (imatrix) | bartowski/Ling-3.1-flash-GGUF | IQ1_S 146.4GB (does not fit a single consumer GPU) |
File sizes of each build:
- Available builds in inclusionAI/Ling-3.1-flash-1019: FP8 534.4GB
- Available builds in bartowski/Ling-3.1-flash-GGUF: IQ1_S 122.0GB / IQ1_M 135.1GB / IQ2_XXS 159.1GB / IQ2_XS 165.2GB / IQ2_S 175.5GB / IQ2_M 198.4GB / Q2_K 204.1GB / IQ3_XXS 230.4GB / IQ3_XS 242.5GB / Q3_K_S 243.6GB / Q3_K_M 256.2GB / Q3_K_L 268.2GB / IQ3_M 276.9GB / IQ4_XS 295.3GB / Q4_0 304.2GB / Q4_K_S 313.5GB / Q4_K_M 333.4GB / IQ4_NL 333.4GB / Q4_1 335.4GB / Q5_K_S 378.2GB / Q5_K_M 406.4GB / Q6_K_S 445.9GB / Q6_K 467.1GB / Q8_0 566.6GB
In addition, 8 converted build(s) from other uploaders exist on Hugging Face; this site lists only builds from the model’s publisher or established quantization maintainers.
This section is appended automatically by Local Model Watch when a converted build of this model appears after publication. Memory figures are estimated from the size of the distributed files. What the quantization names mean: glossary.
Related Articles
- Ling-3.1-flash-1019 Text Generation Model: ~641GB Memory, GGUF Builds
- Ling-3.0-flash-Sante-fp8 Text Generation Model: ~144GB Memory
- ZoomGain-4B: 4GB+ VRAM, GGUF Builds
- ZoomGain-9B Fine-Grained Visual Language Model: 8GB+ VRAM, GGUF Builds
What to Read Next
- Find models by VRAM (This model does not fit even in 80GB; its smallest build needs about 641GB) → VRAM quick reference, including models over 80GB
- Explore the same model family → Ling-3.1 family overview (3 articles, 5 converted builds)
- Engines that run this model → llama.cpp / Ollama / vLLM
- What FP8 mean and where to get this model → GGUF format guide and models / FP8 format guide and models / Quantization and model-format glossary
- Learn about the publisher → inclusionAI (Ant Group): models, licenses and articles
Sources
Update History
- 2026-11-17: Added converted builds to “Quantized and Converted Variants”: inclusionAI/Ling-3.1-flash-1019, bartowski/Ling-3.1-flash-GGUF
- 2026-11-17: Changed the title to show what the article covers (VRAM requirements, file list, etc.).

