Ling-3.0-flash-Sante-fp8 Text Generation Model: ~144GB Memory

Ling-3.0-flash-Sante-fp8 Text Generation Model: ~144GB Memory

At a Glance

Item Value
Repository inclusionAI/Ling-3.0-flash-Sante-fp8
Publisher guide inclusionAI (Ant Group): models and licenses
Published 2026-10-27
License mit
Formats safetensors
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

inclusionAI has released “Ling-3.0-flash-Sante-fp8", an FP8 quantized version of “Ling-3.0-flash-Sante", a large language model specialized in the medical field. The model name “Sante" is derived from the French word for “health", and it is built to handle specialized processing in the healthcare domain while retaining the general-purpose capabilities of its base model, “Ling-3.0-flash". inclusionAI/Ling-3.0-flash-Sante-fp8

This model is designed for specialized medical reasoning, healthcare research, deep research, and advanced tool-enabled workflows. It is intended for use across a wide range of tasks, going beyond simple medical Q&A to include long-document analysis, multi-step research, and multi-agent collaboration. By applying FP8 E4M3 quantization with a block size of 128×128 to expert weights, it reduces the memory footprint of the model weights while maintaining the same model scale as the BF16 version, increasing inference efficiency on compatible hardware. Note that thinking mode is enabled by default in this model.

The publishers note that this model is for research use only and is not a medical device, and that expert verification is required for the use of its outputs.

Specifications

  • Total parameters: 124B
  • Active parameters: 5.1B per token
  • Architecture: Hybrid linear MoE (combination of Kimi Delta Attention (KDA), gated MLA, and sparse MoE layers / BailingMoeV3ForCausalLM)
  • Context length: 256K tokens

Performance

The model card publishes a comparison of specialized healthcare and medical reasoning benchmarks between the BF16 precision “Ling-3.0-flash-Sante" and the FP8 quantized “Ling-3.0-flash-Sante-fp8". The comparison table of measurements conducted by the publisher is as follows:

Benchmark Ling-3.0-flash-Sante (BF16) Ling-3.0-flash-Sante-FP8
Average 73.9 73.9
Healthbench Professional 54.3 52.9
ReDisQA 89.7 90.6
MedXpertQA-Text 51.0 50.5
MedQA-USMLE 94.2 94.5
MedMCQA 80.3 81.1

According to the publisher’s measurements, the overall average score (Average) across all benchmarks is 73.9, maintaining the exact same figure for both the BF16 and FP8 versions. Comparing individual items, the FP8 version recorded slightly higher scores than the BF16 version in three items: ReDisQA (90.6), MedQA-USMLE (94.5), and MedMCQA (81.1). On the other hand, in two items, Healthbench Professional (52.9) and MedXpertQA-Text (50.5), the scores fell below those of the BF16 version.

From the above comparison, although minor numerical changes due to quantization can be seen, high accuracy is maintained overall, and it can be said that the overall performance degradation due to quantization is very minor. Note that these measurements were conducted by the publisher and evaluated using the highest-tier reasoning effort and specified decoding parameters (temperature=1.0, top_p=0.95).

Strengths and Use Cases

Ling-3.0-flash-Sante-fp8 is a model that excels at specialized processing in the medical and healthcare domains. Specific expected use cases include medical and healthcare Q&A, literature search, evidence synthesis, scientific deep research, long-document analysis, structured information extraction, and integration into medical knowledge bases and RAG (Retrieval-Augmented Generation) applications.

It also supports tool-assisted research, multi-agent healthcare workflows, and the development and evaluation of domain-specific AI agents. Because it supports a long context length of 256K tokens, it is suitable for reading long medical documents and handling complex multi-step research tasks. Thinking mode and tool-calling functions are enabled by default and can be utilized through compatible serving frameworks.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch) — 127.5B parameters

Your VRAM Quantization File size Est. memory needed
More than 144GB of VRAM (multi-GPU or CPU offload required) FP8 119.6GB 143.5GB

Inference engine support (architecture name matched against each project’s own model registry in its source code, checked 2026-11-17): llama.cpp: registered, vLLM: registered, MLX (mlx-lm): registered.

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

Not usable in Ollama, LM Studio and llama.cpp yet — we have found no GGUF build.

The publisher ships safetensors only. However, llama.cpp’s registry does list this architecture, so conversion to GGUF is possible and the model will run once someone publishes a converted build. Today it can be run with transformers or vLLM, using the memory figures in the table above.

License — mit (Commercial use allowed): Permits commercial use, modification and redistribution, provided the copyright notice and license text are retained.

Compression: the FP8 build measures 8.06 bits per weight — about 50% the size of the original 16-bit weights, calculated by this site from the actual file sizes.

Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Recent Models in the Same Size Class

Models with over 40B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.

→ Scroll horizontally to see all columns

Model Parameters Smallest VRAM tier License Our article
inclusionAI/Ling-3.1-flash 572.1B — mit Ling-3.1-flash Hybrid Reasoning MoE Model: ~641GB Memory, GGUF Builds (2026-11-17)
inclusionAI/Ling-3.1-flash-1019 572.1B — mit Ling-3.1-flash-1019 Text Generation Model: ~641GB Memory, GGUF Builds (2026-11-17)
inclusionAI/Ling-3.1-flash-base 572.1B — mit Ling-3.1-flash-base Released: 560B Hybrid Linear MoE Model (2026-11-17)
deepseek-ai/DeepSeek-V4-Pro-0813 1650.5B — mit DeepSeek-V4-Pro-0813 Text Generation Model: ~998GB Memory, GGUF Builds (2026-09-26)
nvidia/DeepSeek-V4-Pro-0813-nvfp4-DSpark 1650.5B — mit DeepSeek-V4-Pro-0813-nvfp4-DSpark: ~1005GB Memory (2026-09-10)

How to Get It

  • Distribution format: safetensors
  • License: MIT
  • How to get and supported engines: Can be used using compatible inference engines such as SGLang or vLLM. For SGLang, a dedicated Docker image (lmsysorg/sglang:dev-Ling-3.0-flash) is provided, and for vLLM, builds that include Ling-3.0 support are used.

Model weights are available from Hugging Face, and the official repository ID is inclusionAI/Ling-3.0-flash-Sante-fp8. Note that this model is not a gated model.

Related Articles

What to Read Next

Sources