GPTQ<\/strong> is a post-training method, published in 2022, for quantizing an already-trained LLM to about 3\u20134 bits. It feeds a small amount of calibration text through the model and rounds the weights one layer at a time so that the error introduced by quantization is compensated by the weights not yet quantized<\/strong>. On Hugging Face, GPTQ models come as repositories with Sources: the GPTQ paper (arXiv:2210.17323)<\/a>, vLLM’s quantization documentation<\/a> and the ModelCloud\/GPTQModel README<\/a> (all as of 2026-09-27).<\/em><\/p>\n Local Model Watch has published 1 article(s) on models available in GPTQ: 0 where the repository itself is in GPTQ, and 1 where we found a GPTQ build of the model. The lists below only include builds we have checked (the publisher’s organization and well-known quantizers); a model missing here may still have a GPTQ build elsewhere. Part of our model format index<\/a>.<\/p>\nGPTQ<\/code> in the name (containing .safetensors<\/code> files).<\/p>\nWhy It Matters<\/h2>\n
\n
128g<\/code>); smaller means a smaller file and more quality loss.<\/li>\n<\/ul>\nTips for Running It Locally<\/h2>\n
\n
Our Coverage and Data<\/h2>\n
Main Engines That Load This Format<\/h2>\n