ExLlamaV3 1.4.8 Released: VRAM and FA2 Updates
At a Glance
| Item | Value |
|---|---|
| Repository | turboderp-org/exllamav3 |
| Version | v1.4.8 |
| Published | 2026-09-07 |
| License | MIT |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code at collection time. Dates are JST.
Overview
ExLlamaV3 is a quantization and inference library optimized for running large language models (LLMs) locally on modern consumer GPUs. It is implemented in Python and released under the MIT license.
The latest version, 1.4.8, has been released. The most important changes in this update are the optimization of VRAM (video memory) allocation and the complete removal of code paths related to Flash Attention 2 (FA2). Since the VRAM management method has changed by default in particular, users operating in environments with strict memory usage are encouraged to check the behavior after the update.
Breaking Changes & Deprecations
In this release, all code paths related to FA2 (Flash Attention 2) within the library have been completely removed.
| Item | Old | New |
|---|---|---|
| FA2 code paths | Present | Removed |
This change affects users who have previously configured settings relying on specific FA2 code paths, or who have explicitly utilized features related to FA2. If your existing workflow expects FA2-specific behavior, please pay attention to how it operates.
Key Changes
VRAM Allocation Optimization
The VRAM allocation method has been optimized, switching to use “expandable segments" by default.
This change aims to increase flexibility in memory management during inference execution. For engineers using consumer GPUs, VRAM capacity is an extremely critical resource directly tied to model loading and context length. By making segments expandable, memory allocation efficiency is improved, and operation under limited VRAM environments may become more stable. It is thought to suppress memory fragmentation and enable more efficient resource allocation for dynamic memory requests.
Introduction of Quantization Caches for Specific Models
Caching functionality for the quantization process has been added for the following specific model architectures:
- DSv4-Flash
- Qwen3.8-Flash-Next
- GLM5.3-Flash
Users who frequently quantize and use these models are expected to benefit from reduced time required for quantization work through the use of caches.
Complete Removal of FA2 Code Paths
All code paths related to FA2 (Flash Attention 2) have been completely removed from the library. This is done as part of code base cleanup and optimization, changing it to enhance purity as an inference engine by eliminating unnecessary paths.
Supported Models and Hardware
With this release, caching functionality for the quantization process has been added to specific model architectures. As a result, users utilizing the following models can expect more efficient quantization work:
- DSv4-Flash
- Qwen3.8-Flash-Next
- GLM5.3-Flash
Regarding hardware, this library is optimized to run on modern consumer GPUs. Additionally, PyTorch compatible with CUDA 12.4 or later is required to run it.
How to Get It
There are mainly two installation patterns available:
Installation via PyPI
It can be installed with the following command:
pip install exllamav3
As a note of caution, PyPI packages do not contain pre-built extensions. Therefore, the CUDA Toolkit and an appropriate build environment (such as VS Build Tools on Windows, gcc on Linux, and python-dev headers) must be installed on your system during installation.
Installation from Source
After pre-installing a CUDA-compatible version of PyTorch suited to your environment, follow the steps below. The procedure using CUDA 12.8 is shown as an example.
pip install torch --index-url https://download.pytorch.org/whl/cu128
pip install.
Related Articles
- ExLlamaV3 v1.5.0 Released for Local LLM Inference
- ExLlamaV3 v1.4.9 Released with New Model Support and Fixes
Sources
Update History
- 2026-09-19: Rewrote the article from re-collected sources and restored it from draft to published.