Ling-3.1-flash-base Released: 560B Hybrid Linear MoE Model

At a Glance
| Item | Value |
|---|---|
| Repository | inclusionAI/Ling-3.1-flash-base |
| Family guide | Ling-3.1 guide (3 articles) |
| Publisher guide | inclusionAI (Ant Group): models and licenses |
| Published | 2026-10-25 |
| License | mit |
| Formats | safetensors |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
inclusionAI has released “Ling-3.1-flash-base", a base checkpoint aimed at supporting research and community-driven innovation. It is a hybrid linear Mixture-of-Experts (MoE) model with a total parameter count of 560B and 25B active parameters per token. It is positioned as a foundational model for various research and development tasks such as continued pre-training, domain adaptation, and fine-tuning.
Specifications
- Architecture: Hybrid linear MoE (63 KDA layers + 9 Gated MLA layers)
- Total Parameters: 560B
- Active Parameters per Token: 25B
- Number of Routing Experts: 512 (8 active per token)
- Number of Shared Experts: 1
- Dense Layers: 4
- Attention Heads: 64
- Hidden Size: 4096
- Expert Intermediate Size: 1280
- Dense Intermediate Size: 10240
- Vocabulary Size: 157184
Performance
According to the model card, the evaluation results for Ling-3.1-flash-base are as follows. Measurements for DeepSeek-V4-Pro-Base and MiMo-V2.5-Pro-Base are provided for comparison.
→ Scroll horizontally to see all columns
| Domain | Benchmark | Shot Config | Ling-3.1-flash-base / 560B A25B | DeepSeek-V4-Pro-Base / 1.6T A49B | MiMo-V2.5-Pro-Base / 1.02T A42B |
|---|---|---|---|---|---|
| Language Understanding | Race-High (EM) | 0-shot | 93.88 | 93.37 | 93.60 |
| Language Understanding | SquadV2 (Acc) | 1-shot | 92.79 | 87.70 | 88.93 |
| Language Understanding | Belebele (EM) | 0-shot | 94.67 | 93.83 | 92.72 |
| Language Understanding | C3 (EM) | 0-shot | 83.34 | 81.64 | 81.53 |
| Language Understanding | CCPM (EM) | 0-shot | 91.88 | 93.31 | 90.18 |
| Knowledge | ARC-c (EM) | 0-shot | 97.97 | 96.61 | 96.95 |
| Knowledge | AGIEval (Acc) | few-shot(0~5) | 82.59 | 81.26 | 79.90 |
| Knowledge | SimpleQA-Verified (Acc) | 5-shot | 52.40 | 52.10 | 32.60 |
| Knowledge | C-SimpleQA (Acc) | 5-shot | 77.43 | 77.60 | 70.93 |
| Knowledge | AA-Omniscience (Acc) | 3-shot | 37.67 | 48.16 | 36.45 |
| Knowledge | MMLU-Pro (EM) | 5-shot | 76.71 | 75.28 | 71.26 |
| Knowledge | CEval (EM) | 5-shot | 92.36 | 93.76 | 92.08 |
| Knowledge | CMMLU (EM) | 5-shot | 91.60 | 90.52 | 88.33 |
| Knowledge | GPQA-Diamond (EM) | 0-shot | 56.57 | 52.02 | 54.04 |
| Knowledge | SuperGPQA (EM) | 5-shot | 55.84 | 53.19 | 48.65 |
| Code | HumanEval-Plus (Pass@1) | 0-shot | 79.88 | 57.93 | 82.93 |
| Code | CruxEval (Pass@1) | 1-shot | 80.50 | 69.06 | 71.25 |
| Code | MultiPL-E (Pass@1) | 0-shot | 70.55 | 55.94 | 64.74 |
| Code | MBPP-Plus (Pass@1) | 3-shot | 76.19 | 73.81 | 77.51 |
| Code | BigCodeBench (Pass@1) | 0-shot | 53.60 | 51.49 | 66.75 |
| Code | LCBench 1 (Pass@1) | 3-shot | 68.48 | 59.14 | 67.58 |
| Code | LCBench-cn 1 (Pass@1) | 3-shot | 60.83 | 57.35 | 63.36 |
| Math | MATH500 (Acc) | 4-shot | 79.40 | 68.20 | 57.20 |
| Math | CollegeMath (Acc) | 4-shot | 73.41 | 63.15 | 66.61 |
| Math | FinanceReasoning (Acc) | 3-shot | 81.76 | 82.16 | 80.89 |
| Math | OlympiadBench (Acc) | 3-shot | 60.24 | 46.39 | 48.04 |
| Math | TheoremQA (Acc) | 5-shot | 63.56 | 61.92 | 60.27 |
| Math | OmniMath (Acc) | 3-shot | 65.31 | 34.89 | 40.13 |
| Reasoning | CommonSenseQA (EM) | 5-shot | 92.22 | 86.49 | 88.53 |
| Reasoning | BBH (EM) | 3-shot | 90.25 | 89.98 | 89.80 |
| Reasoning | KorBench (Acc) | 3-shot | 54.96 | 54.56 | 53.20 |
| Reasoning | AutoLogi (Acc) | 3-shot | 67.13 | 64.81 | 66.23 |
| Reasoning | ZebraLogic (Acc) | 1-shot | 30.80 | 20.30 | 28.90 |
| Multilingual | mARC (EM) | 0-shot | 92.98 | 89.10 | 88.55 |
| Multilingual | MMMLU (EM) | 0-shot | 77.56 | 68.81 | 67.15 |
| Multilingual | MultiGSM (Acc) | 4-shot | 87.66 | 85.87 | 86.33 |
| Multilingual | HumanEvalXL (Pass@1) | 0-shot | 83.25 | 70.31 | 87.34 |
| Med | DiagnosisArena (EM) | 0-shot | 62.20 | 55.50 | 52.70 |
| Med | medmcqa (EM) | 5-shot | 78.08 | 80.33 | 77.50 |
| Med | medqa (EM) | 5-shot | 87.82 | 87.75 | 85.55 |
| Long-context | LongBench (Acc) | 0-shot | 51.90 | 17.70 | 15.66 |
| Long-context | LEval (Acc) | 0-shot | 85.09 | 58.34 | 61.49 |
| Long-context | ruler_16k (Acc) | 1-shot | 96.90 | 97.00 | 90.61 |
| Long-context | ruler_32k (Acc) | 1-shot | 95.87 | 95.57 | 89.10 |
| Long-context | ruler_64k (Acc) | 1-shot | 93.81 | 95.56 | 87.19 |
| Long-context | ruler_128k (Acc) | 1-shot | 88.64 | 95.06 | 85.79 |
| Long-context | ruler_256k (Acc) | 1-shot | 86.64 | 94.37 | 86.00 |
Based on the publication’s evaluation table, Ling-3.1-flash-base shows strong performance in long-context understanding and mathematical reasoning domains. It records scores of 51.90 on LongBench and 85.09 on LEval, significantly outperforming larger comparison models in measuring long document summarization and comprehension. It also takes the top spot with 79.40 on MATH500, which measures high school to competition-level math proficiency. On the other hand, strengths and weaknesses vary by field, as it falls behind other models in knowledge-based QA tasks such as AA-Omniscience and scores 53.60 on BigCodeBench, which measures practical complex coding capabilities.
Strengths and Use Cases
Ling-3.1-flash-base is not a finalized model intended for direct end-user interaction, but rather a base model released to support additional learning, research, and development by the community and engineers. The model card recommends its use in the following areas:
Recommended Use Cases
- Continued Pre-training and Mid-training: Expanding and enhancing the base model with proprietary corpora or additional datasets.
- Supervised Fine-Tuning (SFT) for Domain Adaptation: Fine-tuning to specialize the model in specific operational domains or specialized fields.
- Post-Training Research: Utilization as an experimental foundation for preference optimization or reinforcement learning.
- Distillation Research: Research on transferring knowledge from this large-scale model to smaller models.
- Long-Context and MoE System Research: Research and development regarding the control of the hybrid linear attention structure combining 63 KDA layers and 9 Gated MLA layers, as well as the sparse MoE architecture with 512 routing experts.
Not Recommended For
- Direct Chat Deployment: Deploying it directly into conversational services for end-users without alignment tuning or safety evaluation.
- Safety-Critical Applications: Utilizing it for high-risk tasks without task-specific safety verification or alignment.
- Production Deployment Without Verification: Deploying it to production environments without task-specific evaluation or post-training.
While it shows high performance in long-context understanding and mathematical reasoning, because it is a pre-trained base checkpoint, appropriate post-training according to the objective is required if conversational ability or specific output formats are expected.
Hardware Requirements
Estimated requirements (calculated by Local Model Watch) — 572.1B parameters
| Your VRAM | Quantization | File size | Est. memory needed |
|---|---|---|---|
| More than 1279GB of VRAM (multi-GPU or CPU offload required) | F32 | 1065.7GB | 1278.8GB |
Inference engine support (architecture name matched against each project’s own model registry in its source code, checked 2026-11-17): llama.cpp: registered, vLLM: registered, MLX (mlx-lm): registered.
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. The listed files are distributed in F32 (32-bit). Loading in a lower precision (BF16 or quantized) needs less memory, so treat this as the size of the files as distributed, not a minimum. This site has not measured the lower-precision figures. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.
Can You Run It Locally?
Not usable in Ollama, LM Studio and llama.cpp yet — we have found no GGUF build.
The publisher ships safetensors only. However, llama.cpp’s registry does list this architecture, so conversion to GGUF is possible and the model will run once someone publishes a converted build. Today it can be run with transformers or vLLM, using the memory figures in the table above.
License — mit (Commercial use allowed): Permits commercial use, modification and redistribution, provided the copyright notice and license text are retained.
Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.
Recent Models in the Same Size Class
Models with over 40B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.
→ Scroll horizontally to see all columns
| Model | Parameters | Smallest VRAM tier | License | Our article |
|---|---|---|---|---|
| inclusionAI/Ling-3.1-flash | 572.1B | — | mit | Ling-3.1-flash Hybrid Reasoning MoE Model: ~641GB Memory, GGUF Builds (2026-11-17) |
| inclusionAI/Ling-3.1-flash-1019 | 572.1B | — | mit | Ling-3.1-flash-1019 Text Generation Model: ~641GB Memory, GGUF Builds (2026-11-17) |
| inclusionAI/Ling-3.0-flash-Sante-fp8 | 127.5B | — | mit | Ling-3.0-flash-Sante-fp8 Text Generation Model: ~144GB Memory (2026-11-17) |
| deepseek-ai/DeepSeek-V4-Pro-0813 | 1650.5B | — | mit | DeepSeek-V4-Pro-0813 Text Generation Model: ~998GB Memory, GGUF Builds (2026-09-26) |
| nvidia/DeepSeek-V4-Pro-0813-nvfp4-DSpark | 1650.5B | — | mit | DeepSeek-V4-Pro-0813-nvfp4-DSpark: ~1005GB Memory (2026-09-10) |
How to Get It
The model weights for Ling-3.1-flash-base are available in the Hugging Face repository (inclusionAI/Ling-3.1-flash-base).
While FP8 precision was used during pre-training, the published checkpoint consists of BF16 precision weights created via Warmup-Stable and Merge (WSM) checkpoint merging. The file format is provided in safetensors format.
Regarding usage restrictions, the repository does not have a gated access requirement (gated: false), allowing anyone to access and obtain it directly. Additionally, the MIT license is applied.
For specific implementation examples and code for fine-tuning, the official ling-cookbook is provided as a guide.
Related Articles
- Ling-3.1-flash-1019 Text Generation Model: ~641GB Memory, GGUF Builds
- Ling-3.1-flash Hybrid Reasoning MoE Model: ~641GB Memory, GGUF Builds
- Duo-1.0-distill-quant FP8 Quantized Realization Model: 80GB+ VRAM
- Duo-1.2-causal-base Released for Real-time Multimodal Models
What to Read Next
- Find models by VRAM (This model does not fit even in 80GB; its smallest build needs about 1279GB) → VRAM quick reference, including models over 80GB
- Explore the same model family → Ling-3.1 family overview (3 articles, 5 converted builds)
- Engines that run this model → vLLM
- Formats this model is available in → Safetensors format guide and models
- Learn about the publisher → inclusionAI (Ant Group): models, licenses and articles

