{"id":86,"date":"2026-09-07T08:11:27","date_gmt":"2026-09-06T23:11:27","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/09\/07\/exllamav3-1-4-8-released\/"},"modified":"2026-09-20T17:37:07","modified_gmt":"2026-09-20T08:37:07","slug":"exllamav3-1-4-8-released","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/07\/exllamav3-1-4-8-released\/","title":{"rendered":"ExLlamaV3 1.4.8 Released: VRAM and FA2 Updates"},"content":{"rendered":"

<\/p>\n

At a Glance<\/h2>\n
\n\n\n\n\n\n\n\n\n\n
Item<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n
Repository<\/td>\nturboderp-org\/exllamav3<\/a><\/td>\n<\/tr>\n
Version<\/td>\nv1.4.8<\/a><\/td>\n<\/tr>\n
Published<\/td>\n2026-09-07<\/td>\n<\/tr>\n
License<\/td>\nMIT<\/td>\n<\/tr>\n
Source type<\/td>\nPrimary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Values determined by this site’s code at collection time. Dates are JST.<\/em><\/p>\n

<\/p>\n

Overview<\/h2>\n

ExLlamaV3 is a quantization and inference library optimized for running large language models (LLMs) locally on modern consumer GPUs. It is implemented in Python and released under the MIT license.<\/p>\n

The latest version, 1.4.8, has been released. The most important changes in this update are the optimization of VRAM (video memory) allocation and the complete removal of code paths related to Flash Attention 2 (FA2). Since the VRAM management method has changed by default in particular, users operating in environments with strict memory usage are encouraged to check the behavior after the update.<\/p>\n

Breaking Changes & Deprecations<\/h2>\n

In this release, all code paths related to FA2 (Flash Attention 2) within the library have been completely removed.<\/p>\n

\n\n\n\n\n\n
Item<\/th>\nOld<\/th>\nNew<\/th>\n<\/tr>\n<\/thead>\n
FA2 code paths<\/td>\nPresent<\/td>\nRemoved<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

This change affects users who have previously configured settings relying on specific FA2 code paths, or who have explicitly utilized features related to FA2. If your existing workflow expects FA2-specific behavior, please pay attention to how it operates.<\/p>\n

Key Changes<\/h2>\n

VRAM Allocation Optimization<\/h3>\n

The VRAM allocation method has been optimized, switching to use “expandable segments” by default.<\/p>\n

This change aims to increase flexibility in memory management during inference execution. For engineers using consumer GPUs, VRAM capacity is an extremely critical resource directly tied to model loading and context length. By making segments expandable, memory allocation efficiency is improved, and operation under limited VRAM environments may become more stable. It is thought to suppress memory fragmentation and enable more efficient resource allocation for dynamic memory requests.<\/p>\n

Introduction of Quantization Caches for Specific Models<\/h3>\n

Caching functionality for the quantization process has been added for the following specific model architectures:<\/p>\n