NVFP4 / MXFP4 Model Format Explained: Supported Engines and Models
What Are NVFP4 and MXFP4?
NVFP4 and MXFP4 store weights as 4-bit floating-point numbers. Four bits can express very few values, so both split the weights into small blocks and give each block its own scale factor to recover precision.
- MXFP4 is one of the “Microscaling (MX)" formats standardized by the Open Compute Project (OCP), with one scale per 32 values. It became widely known when OpenAI released the MoE weights of gpt-oss in this format.
- NVFP4 is NVIDIA’s format. It keeps finer-grained scales than MXFP4 — one FP8 scale per 16 values — for better accuracy. NVIDIA and others publish NVFP4 versions of large models.
On Hugging Face these usually come as repositories with NVFP4 or MXFP4 in the name (containing .safetensors files).
Why It Matters
- About a quarter of the original BF16 size. Huge models fit on fewer GPUs.
- Newer GPUs compute in 4-bit directly. NVIDIA’s Blackwell generation (B200, RTX 50 series and others) has hardware support for 4-bit floating-point math, which also helps speed.
- Older GPUs may still load it. vLLM supports generations such as Ampere by keeping only the weights in 4-bit and computing in 16-bit (Marlin); its table notes that Turing does not support Marlin MXFP4.
Tips for Running It Locally
- Mainly run with vLLM, SGLang and TensorRT-LLM. Ollama and LM Studio cannot load these
.safetensorsfiles. llama.cpp does handle MXFP4 GGUF files for gpt-oss, so for gpt-oss you can choose a GGUF build. - The same name can mean different coverage. It is common to quantize only the MoE experts to 4-bit and keep the rest in FP8 or BF16. Check the memory requirements in each article’s table, which uses the actual file sizes.
Sources: vLLM’s quantization documentation (hardware compatibility table), NVIDIA’s technical blog “Introducing NVFP4", the OCP Microscaling Formats (MX) v1.0 specification and the openai/gpt-oss-20b model card (all as of 2026-09-27).
Our Coverage and Data
Local Model Watch has published 6 article(s) on models available in NVFP4 / MXFP4: 3 where the repository itself is in NVFP4 / MXFP4, and 3 where we found a NVFP4 / MXFP4 build of the model. The lists below only include builds we have checked (the publisher’s organization and well-known quantizers); a model missing here may still have a NVFP4 / MXFP4 build elsewhere. Part of our model format index.
Main Engines That Load This Format
| Engine | Overview |
|---|---|
| vLLM | High-throughput LLM serving engine built around PagedAttention. Handles concurrent requests and quantized weights (AWQ / GPTQ / FP8) and is widely used on server-class GPUs. |
| SGLang | LLM serving framework known for RadixAttention prefix caching and structured output. A common alternative to vLLM for serving. |
Models Available in NVFP4 / MXFP4
→ Scroll horizontally to see all columns
| Published | Model | Where to get it | Quantizations | Smallest VRAM tier | Article |
|---|---|---|---|---|---|
| 2026-09-27 | XiaomiMiMo/MiMo-V2.6-Pro-RL | mlx-community/MiMo-V2.6-Pro-RL-mxfp4-q8 | MLX 4bit | — | Xiaomi Releases MiMo-V2.6-Pro-RL: 1.02T MoE Flagship |
| 2026-09-26 | Qwen/Qwen3.8-27B | unsloth/Qwen3.8-27B-NVFP4 | NVFP4 | 32GB | Qwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds |
| 2026-09-23 | IFM/K2-Horizon-32B-NVFP4 | This repository | — | 32GB | K2-Horizon-32B-NVFP4 Long-Context Reasoning Model: 32GB+ VRAM |
| 2026-09-23 | IFM/K2-Horizon-375B-A23B-NVFP4 | This repository | — | — | K2-Horizon-375B-A23B-NVFP4 Text Generation Model: ~257GB Memory |
| 2026-09-10 | nvidia/DeepSeek-V4-Pro-0813-nvfp4-DSpark | This repository | — | — | DeepSeek-V4-Pro-0813-nvfp4-DSpark: ~1005GB Memory |
| 2026-09-10 | deepseek-ai/DeepSeek-V4.1-Flash | nvidia/DeepSeek-V4.1-Flash-NVFP4 | — | — | DeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory |
“Smallest VRAM tier" is the smallest tier in the requirements table of each article (for other builds, of those builds; for image, video and audio models, always the article’s own table, which counts every component). Leave headroom for context length.
Last updated 2026-09-27 (JST). The explanation at the top of this page was written with the help of AI from the primary sources it cites. The tables under “Our Coverage and Data" are assembled by code from our article log.