Ling-3.1-flash Hybrid Reasoning MoE Model: ~641GB Memory, GGUF Builds

Ling-3.1-flash Hybrid Reasoning MoE Model: ~641GB Memory, GGUF Builds

At a Glance

Item Value
Repository inclusionAI/Ling-3.1-flash
Family guide Ling-3.1 guide (3 articles)
Publisher guide inclusionAI (Ant Group): models and licenses
Published 2026-10-13
License mit
Formats safetensors
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

inclusionAI has released the hybrid reasoning model “Ling-3.1-flash". It is positioned as an upgraded version based on the previous generation Ling-3.1-flash, with improvements to its architecture and performance on practical tasks. It is a MoE model with 560B total parameters and 25B active parameters, supporting a long context length of 1M tokens, and is designed with real-world tasks such as agentic tasks, software development, and office work in mind.

Specifications

The configuration listed in the model card is as follows:

  • Total Parameters: 560B
  • Active Parameters (per token): 25B
  • Architecture: Hybrid-linear MoE
  • Context Length: 1M tokens
  • Transformer Layer Configuration: KDA 63 layers + Gated MLA 9 layers (7:1 ratio)
  • Dense Layers: 4
  • Routed Experts: 512
  • Shared Experts: 1
  • Activated Experts (per token): 8
  • Attention Heads: 64
  • Hidden Size: 4096
  • Expert Intermediate Size: 1280
  • Dense Intermediate Size: 10240
  • Vocabulary Size: 157184

Performance

The model card does not include a comparative table showing specific scores, and evaluation results are mentioned only in text. According to the publishers, Ling-3.1-flash shows strong performance on coding and agent benchmarks such as Terminal-Bench 4.0, SWE-bench Pro, FrontierSWE, GDPVal-AA V2.1, Automationbench, MCP-Atlas, and SkillsBench.

Among those whose contents can be confirmed via benchmark_glossary, Terminal-Bench is a metric measuring whether commands can be actually executed in a terminal to complete given tasks using Pass@1(%), and SWE-bench is a metric measuring whether actual GitHub issues can be resolved by editing repositories using Resolved (percentage of solved problems, %). Both are metrics where higher is better, and the publishers state that it performs “strongly" on benchmark groups including these, though specific numbers cannot be confirmed from the materials.

Additionally, it is reported that good user experiences can be obtained when used on actual agentic frameworks such as Claude Code, Kilo Code, Qwen Code, Hermes Agent, and OpenClaw, and it is explained that it exhibits strong performance on practical tasks in fields such as STEM, medicine, and finance beyond coding agent use cases. Furthermore, for long-term tasks involving long durations and hundreds of tool calls, a proprietary harness called “LingCode" based on the open-source coding agent “Pi" is introduced, though this is an ingenuity on the execution environment side rather than the model itself, and open-sourcing is reportedly under consideration for the future. Note that these are all evaluations and explanations by the publishers themselves and are not third-party verification results.

Strengths and Use Cases

Ling-3.1-flash is designed with a strong focus on agent use cases that autonomously handle complex tasks in the real world. The model card lists general-purpose agents, search, office productivity enhancement, and software engineering as its main applications.

Specifically, it is suited for tasks involving tool execution and command operations using a terminal, and task automation through agent frameworks such as Claude Code and OpenClaw. Furthermore, performance improvements are mentioned in domains requiring advanced specialized knowledge such as general STEM fields, medicine, finance, and materials science. Because it features a wide context window reaching 1M tokens, it is suited for workflows where maintaining context is essential, such as reading lengthy documents or long-term coding tasks repeating numerous tool calls.

In addition, an option to enable the thinking process in the chat template (enable_thinking) is provided, and utilization in logical tasks and problem-solving solved step-by-step with reasoning in between is also envisioned.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch) — 572.1B parameters

Your VRAM Quantization File size Est. memory needed
More than 641GB of VRAM (multi-GPU or CPU offload required) FP8 534.4GB 641.3GB

Inference engine support (architecture name matched against each project’s own model registry in its source code, checked 2026-11-17): llama.cpp: registered, vLLM: registered, MLX (mlx-lm): registered.

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

Runs in Ollama, LM Studio and llama.cpp via a converted build.

The publisher ships safetensors, but bartowski/Ling-3.1-flash-GGUF provides a GGUF build you can use.

License — mit (Commercial use allowed): Permits commercial use, modification and redistribution, provided the copyright notice and license text are retained.

Compression: the FP8 build measures 8.02 bits per weight — about 50% the size of the original 16-bit weights, calculated by this site from the actual file sizes.

Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Recent Models in the Same Size Class

Models with over 40B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.

→ Scroll horizontally to see all columns

Model Parameters Smallest VRAM tier License Our article
inclusionAI/Ling-3.1-flash-1019 572.1B — mit Ling-3.1-flash-1019 Text Generation Model: ~641GB Memory, GGUF Builds (2026-11-17)
inclusionAI/Ling-3.1-flash-base 572.1B — mit Ling-3.1-flash-base Released: 560B Hybrid Linear MoE Model (2026-11-17)
inclusionAI/Ling-3.0-flash-Sante-fp8 127.5B — mit Ling-3.0-flash-Sante-fp8 Text Generation Model: ~144GB Memory (2026-11-17)
deepseek-ai/DeepSeek-V4-Pro-0813 1650.5B — mit DeepSeek-V4-Pro-0813 Text Generation Model: ~998GB Memory, GGUF Builds (2026-09-26)
nvidia/DeepSeek-V4-Pro-0813-nvfp4-DSpark 1650.5B — mit DeepSeek-V4-Pro-0813-nvfp4-DSpark: ~1005GB Memory (2026-09-10)

How to Get It

The weights of Ling-3.1-flash are published on Hugging Face in FP8 safetensors format, and the license is the MIT License. Gate application (individual consent to terms of use) is not required for use, and it can be downloaded and used directly.

As execution engines, setup procedures are introduced for two types: SGLang, which is guided as the recommended environment, and vLLM.

Running with SGLang (Recommended)

An official Docker image containing support for Ling-3.1-flash, MTP (NEXTN) speculative decoding, and decode optimizations for Hopper is provided.

Pulling the image:

docker pull lmsysorg/sglang:dev-ling-3.1-flash

Starting the server (e.g., specifying tensor parallelism degree 8 and expert parallelism degree 8 in an 8x H200 environment):

docker run --gpus all --network host --ipc=host --rm \
  lmsysorg/sglang:dev-ling-3.1-flash \
  python3 -m sglang.launch_server \
    --model-path inclusionAI/Ling-3.1-flash \
    --tp 8 --ep 8 \
    --speculative-algorithm NEXTN \
    --host 0.0.0.0 --port 30000

Note that for a configuration using 4 GB300 units, specify --tp 4 --ep 4. In addition, decoding speedup through Hopper optimizations can be enabled by supplying the environment variables SGLANG_ENABLE_FP8_BLOCK_GEMV=1 and SGLANG_ENABLE_FUSED_KDA_OUTPUT_QUANT=1.

Example inference request from a client:

curl -s http://localhost:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "Ling-3.1-flash",
       "messages": [{"role": "user", "content": "What is the capital of France?"}],
       "chat_template_kwargs": {"enable_thinking": true},
       "stream": true,
       "temperature": 1.0,
       "top_k": 20,
       "top_p": 0.95}'

Running with vLLM

When using vLLM, building a virtual environment using uv and building from the vLLM repository are guided.

Environment setup procedure:

pip install uv

uv venv ~/my_ling_env

source ~/my_ling_env/bin/activate

git clone -b vllm https://github.com/vllm-project/vllm.git

cd vllm

VLLM_USE_PRECOMPILED=1 uv pip install --editable. --torch-backend=auto

Starting the server:

vllm serve "$MODEL_PATH" \
    --port "$PORT" \
    --trust-remote-code \
    --served-model-name auto \
    --tensor-parallel-size 8 \
    --gpu-memory-utilization 0.85 \
    --enable-prefix-caching \
    --enable-expert-parallel \
    --mamba-cache-mode align \
    --enable-auto-tool-choice \
    --tool-call-parser ling3 \
    --reasoning-parser ling3 \
    --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

Example inference request from a client:

curl -s http://${MASTER_IP}:${PORT}/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "auto", 
       "messages": [{"role": "user", "content": "What is the capital of France?"}],
       "chat_template_kwargs": {"enable_thinking": true},
       "stream": true,
       "temperature": 1.0, 
       "top_k": 20,
       "top_p": 0.95}'

As sampling parameters, a combination of temperature=1.0, top_p=0.95, and top_k=20 is recommended, and enabling enable_thinking is recommended to fully bring out performance.

Quantized and Converted Variants

→ Scroll horizontally to see all columns

Added Publisher Format Repository Smallest VRAM tier (build, est. memory)
2026-11-17 inclusionAI FP8 inclusionAI/Ling-3.1-flash-1019 FP8 641.3GB (does not fit a single consumer GPU)
2026-11-17 bartowski GGUF (imatrix) bartowski/Ling-3.1-flash-GGUF IQ1_S 146.4GB (does not fit a single consumer GPU)

File sizes of each build:

  • Available builds in inclusionAI/Ling-3.1-flash-1019: FP8 534.4GB
  • Available builds in bartowski/Ling-3.1-flash-GGUF: IQ1_S 122.0GB / IQ1_M 135.1GB / IQ2_XXS 159.1GB / IQ2_XS 165.2GB / IQ2_S 175.5GB / IQ2_M 198.4GB / Q2_K 204.1GB / IQ3_XXS 230.4GB / IQ3_XS 242.5GB / Q3_K_S 243.6GB / Q3_K_M 256.2GB / Q3_K_L 268.2GB / IQ3_M 276.9GB / IQ4_XS 295.3GB / Q4_0 304.2GB / Q4_K_S 313.5GB / Q4_K_M 333.4GB / IQ4_NL 333.4GB / Q4_1 335.4GB / Q5_K_S 378.2GB / Q5_K_M 406.4GB / Q6_K_S 445.9GB / Q6_K 467.1GB / Q8_0 566.6GB

In addition, 8 converted build(s) from other uploaders exist on Hugging Face; this site lists only builds from the model’s publisher or established quantization maintainers.

This section is appended automatically by Local Model Watch when a converted build of this model appears after publication. Memory figures are estimated from the size of the distributed files. What the quantization names mean: glossary.

Related Articles

What to Read Next

Sources

Update History

  • 2026-11-17: Added converted builds to “Quantized and Converted Variants”: inclusionAI/Ling-3.1-flash-1019, bartowski/Ling-3.1-flash-GGUF
  • 2026-11-17: Changed the title to show what the article covers (VRAM requirements, file list, etc.).