ExLlamaV3 1.5.0 Released: Major MoE Inference Speedups

September 13, 2026September 20, 2026

At a Glance

Values determined by this site’s code at collection time. Dates are JST.

Overview

ExLlamaV3 is a quantization and inference library optimized for running large language models (LLMs) locally on modern consumer GPUs. It is implemented in Python and provided under the MIT license.

The latest version, 1.5.0, has been released. The primary highlight of this update is a significant boost in inference performance for MoE (Mixture of Experts) models. In particular, improvements to prefill and decode speeds for MoE models make it possible to run more advanced models at practical speeds even in resource-constrained local environments.

Key Changes

Improved MoE Inference Performance and Speculative Decoding

Processing speeds for prefill and decode have been enhanced in models employing the MoE (Mixture of Experts) architecture. Furthermore, the efficiency of speculative decoding for MoE models has been improved.

This change directly benefits users running MoE models such as Gemma4-26B-A3B and Qwen3.8-Flash-Next in local environments. According to the provided benchmarks, running Gemma4-26B-A3B on an RTX 5090 confirms a 22% speedup in prefill and a 10% speedup in decode. Improvements of 26% in prefill and 4% in decode are also seen on the RTX 3090 environment. For Qwen3.8-Flash-Next, using a Pro 6000 is reported to achieve a 27% speedup in prefill and a 14% speedup in decode, promising a better felt experience when using MoE models. Users can enjoy these improvements simply by updating the library.

Experimental CPU Offload Acceleration for Linux

For users in Linux environments, an experimental “zero-copy pinned arena mode" has been introduced for CPU offloading.

This mechanism increases data transfer efficiency when offloading model layers to system memory (RAM) to run when GPU video memory (VRAM) is insufficient. This feature is particularly important for users trying to run large models while using GPUs with limited VRAM capacity. Benchmarks show extremely high performance improvements when running Qwen3.8-Flash-Next with 80% CPU offload (Pro 6000 + TR 7960X), with prefill improving by 54% and decode by 20%. However, please note that this is currently Linux-only and remains in an experimental stage.

Faster Quantization Process

The model quantization process has been accelerated.

This directly streamlines workflows for engineers who quantize and use models in their own environments or frequently generate new quantization formats. Because wait times for preparing models are drastically reduced, it is possible to speed up the trial-and-error cycle for new models. Users can benefit from this by re-running the quantization process using the latest library.

Nemotron-3-Super Support

Fixes to NemotronHForCausalLM were made, officially adding support for the Nemotron-3-Super model.

Users utilizing models in the Nemotron series can apply this update to properly run the latest Nemotron models within the ExLlamaV3 environment. There are no changes to existing model usage procedures, but an update is required to use the latest models.

Supported Models and Hardware

With this update, improvements and support have been confirmed for the following model architectures and environments.

Newly Supported Models

• Nemotron-3-Super (supported via fixes to NemotronHForCausalLM)

Models with Confirmed Performance Improvements

• Gemma4-26B-A3B (MoE model)
• Qwen3.8-Flash-Next (MoE model)
• GLM5.3-Flash
• Qwen3.8-27B
• gpt-oss-20b

Hardware and Environment Impact

• Linux Users: Significant speedups can be expected when using CPU offload via “zero-copy pinned arena mode".
• NVIDIA GPU Users: Improvements in inference speed for MoE and dense models are reported on various consumer GPUs such as the RTX 5090, 4090, 3090Ti, 3090, and Pro 6000.

How to Get It

Installing ExLlamaV3 requires a pre-installed compatible version of PyTorch (CUDA 12.4 or later). Typical installation steps are as follows.

When Using PyPI

pip install exllamav3

Note: Because PyPI packages do not include pre-built extensions, a CUDA Toolkit and build environment (VS Build Tools on Windows, gcc on Linux, etc.) are required.

When Installing from Source

pip install torch --index-url https://download.pytorch.org/whl/cu128
pip install.

Related Articles

• ExLlamaV3 v1.4.9 Released with New Model Support and Fixes
• exllamav3 v1.4.8 Released: VRAM and Cache Updates

Sources

• turboderp-org/exllamav3 1.5.0 (GitHub Release)

Update History

• 2026-09-20: Rewrote the article from re-collected sources and restored it from draft to published.