{"id":10625,"date":"2026-11-17T10:59:37","date_gmt":"2026-11-17T01:59:37","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/?p=10625"},"modified":"2026-11-17T11:09:02","modified_gmt":"2026-11-17T02:09:02","slug":"exllamav3-v1-6-0-released-2","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/exllamav3-v1-6-0-released-2\/","title":{"rendered":"exllamav3 v1.6.0 Released"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/github.com\/turboderp-org\/exllamav3\">turboderp-org\/exllamav3<\/a><\/td>\n<\/tr>\n<tr>\n<td>Version<\/td>\n<td><a href=\"https:\/\/github.com\/turboderp-org\/exllamav3\/releases\/tag\/v1.6.0\">v1.6.0<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-10-08<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>MIT<\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>Version 1.6.0 of turboderp-org\/exllamav3 was released on GitHub on October 7, 2026. exllamav3 is a quantization and inference library optimized for running LLMs locally on the latest generation of consumer GPUs, implemented in Python and available under the MIT license. Featuring proprietary quantization methods and kernels following the exl2 format, it is a project used by engineers who want to efficiently run large language models on their local GPUs.<\/p>\n<p>Regarding this release, although the v1.6.0 tag and release page have been created on GitHub, the release notes body has not been provided at this time, and specific changes cannot be confirmed from the documentation. Therefore, it is unverified as to &#8220;what has changed&#8221; or &#8220;whether one should upgrade&#8221; in this version. Existing users are advised to check the release page and recent commit history themselves before updating.<\/p>\n<h2>Key Changes<\/h2>\n<p>No release notes body or individual change items (new features, performance improvements, bug fixes) are included in the documentation provided for this release. Consequently, specific details such as which features have been added or which models and hardware are affected cannot be confirmed at this time.<\/p>\n<p>Since exllamav3 is a core quantization and inference library, it is generally possible that version updates could affect kernel behavior or quantization processing, but nothing specific to v1.6.0 is stated in the documentation, and this article does not make speculative claims. If you want to know the specific changes, please check the release page below or the repository&#8217;s commit history and issue tracker directly.<\/p>\n<h2>How to Get It<\/h2>\n<p>When installing exllamav3, you need to install a CUDA-compatible PyTorch (CUDA 12.4 or later) tailored to your environment beforehand. Note that Torch dependencies are not automatically handled by <code>pip<\/code>, requiring careful attention during environment setup.<\/p>\n<p>Typical installation procedures include installing via PyPI and installing directly from source.<\/p>\n<h3>Installation from PyPI<\/h3>\n<p>To install the package published on PyPI, run the following command.<\/p>\n<pre><code class=\"language-sh\">pip install exllamav3\n<\/code><\/pre>\n<p>Note that the PyPI package does not include pre-built binary extensions. Therefore, the CUDA toolkit and various tools required for building (such as gcc and <code>python-dev<\/code> headers on Linux, or Visual Studio Build Tools on Windows) must be installed in advance.<\/p>\n<h3>Installing CUDA-Compatible PyTorch and Manual Installation<\/h3>\n<p>If you want to install from source after installing the CUDA version of PyTorch matching your environment, install PyTorch by specifying the wheel index and then install the package as follows.<\/p>\n<pre><code class=\"language-sh\">pip install torch --index-url https:\/\/download.pytorch.org\/whl\/cu128\npip install.\n<\/code><\/pre>\n<h3>Setting Build Environment Variables<\/h3>\n<p>When building C++\/CUDA extensions, you can set the following environment variables depending on your environment:<\/p>\n<ul>\n<li><code>MAX_JOBS<\/code>: Specified to limit the number of parallel jobs to an appropriate value like 4 when parallel build processes by ninja cause the system to run out of memory.<\/li>\n<li><code>EXLLAMA_NOCOMPILE<\/code>: Set when installing the library without pre-compiling C++\/CUDA extensions. In this case, PyTorch builds and loads the extensions at runtime.<\/li>\n<\/ul>\n<p>After completing the installation, you can run model conversion scripts and other tools from the main repository directory.<\/p>\n<pre><code class=\"language-sh\">python convert.py -i...\n<\/code><\/pre>\n<p><!-- lmw:releases --><\/p>\n<h2>Releases Since Our Last Article<\/h2>\n<p><em>Compiled by Local Model Watch from the project&#8217;s GitHub releases: the versions between this release and the last one we covered, which did not get separate articles.<\/em> <em>Full history: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-exllamav3-en\/\">release tracker<\/a>.<\/em><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Version<\/th>\n<th>Released<\/th>\n<th>Release notes<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>v1.5.4<\/td>\n<td>2026-10-04<\/td>\n<td><a href=\"https:\/\/github.com\/turboderp-org\/exllamav3\/releases\/tag\/v1.5.4\">GitHub<\/a><\/td>\n<\/tr>\n<tr>\n<td>v1.5.3<\/td>\n<td>2026-09-28<\/td>\n<td><a href=\"https:\/\/github.com\/turboderp-org\/exllamav3\/releases\/tag\/v1.5.3\">GitHub<\/a><\/td>\n<\/tr>\n<tr>\n<td>v1.5.2<\/td>\n<td>2026-09-26<\/td>\n<td><a href=\"https:\/\/github.com\/turboderp-org\/exllamav3\/releases\/tag\/v1.5.2\">GitHub<\/a><\/td>\n<\/tr>\n<tr>\n<td>v1.5.1<\/td>\n<td>2026-09-21<\/td>\n<td><a href=\"https:\/\/github.com\/turboderp-org\/exllamav3\/releases\/tag\/v1.5.1\">GitHub<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><!-- \/lmw:releases --><\/p>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/07\/exllamav3-1-4-8-released\/\">ExLlamaV3 1.4.8 Released: VRAM and FA2 Updates<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/13\/exllamav3-v1-5-0-released\/\">ExLlamaV3 1.5.0 Released: Major MoE Inference Speedups<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/10\/exllamav3-v149-released\/\">ExLlamaV3 v1.4.9 Released: Faster CPU Offloading<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Follow this tool<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-exllamav3-en\/\">ExLlamaV3 overview and release history (71 releases tracked)<\/a><\/li>\n<li><strong>Other quantization, model formats and fine-tuning<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-ggml-en\/\">ggml<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-unsloth-en\/\">Unsloth<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/github.com\/turboderp-org\/exllamav3\/releases\/tag\/v1.6.0\">GitHub Release: turboderp-org\/exllamav3 v1.6.0<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Version 1.6.0 of turboderp-org\/exllamav3 has been released on GitHub. Read about installation methods and requirements.<\/p>\n","protected":false},"author":1,"featured_media":10624,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[105],"tags":[150,163,154,1547,1193,526],"class_list":["post-10625","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-engines-and-tools","tag-exllamav3-en","tag-gguf-en","tag-turboderp-en","tag-verified","tag--en"],"lang":"en","translations":{"en":10625,"ja":10623},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/10625","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=10625"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/10625\/revisions"}],"predecessor-version":[{"id":17719,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/10625\/revisions\/17719"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/10624"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=10625"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=10625"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=10625"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}