AWQ Model Format Explained: Supported Engines and Models
What Is AWQ?
AWQ (Activation-aware Weight Quantization) is a method for shrinking LLM weights to about 4 bits, published by an MIT research team in 2023. On Hugging Face, models quantized with it are distributed as repositories with AWQ in the name (containing .safetensors files).
Why It Matters
- 4-bit with little quality loss. It finds the small set of weight channels that matter most given the size of the inputs (activations) and scales the weights to protect them. The paper shows this loses less quality than rounding every weight the same way.
- Fast on GPUs. It quantizes weights only (W4A16) and has dedicated compute kernels.
- Supported out of the box by serving engines. vLLM and SGLang load it with almost the same steps as the original model.
Tips for Running It Locally
- It assumes an NVIDIA GPU. In vLLM’s compatibility table, AWQ targets Turing (RTX 20 series) and newer. On CPU or a Mac, choose GGUF or MLX.
- Ollama and LM Studio (llama.cpp-based) cannot load it.
- Best for serving several users or running an API server. For one person chatting, GGUF offers a wider choice of quantization levels and is easier to set up.
- AutoAWQ, long used to create AWQ models, is no longer developed, and vLLM points to successor tools such as LLM Compressor. This does not matter if you only use already-quantized models.
Sources: the AWQ paper (arXiv:2306.00978) and vLLM’s quantization documentation (both as of 2026-09-27).
Our Coverage and Data
Local Model Watch has published 1 article(s) on models available in AWQ: 1 where the repository itself is in AWQ, and 0 where we found a AWQ build of the model. The lists below only include builds we have checked (the publisher’s organization and well-known quantizers); a model missing here may still have a AWQ build elsewhere. Part of our model format index.
Main Engines That Load This Format
| Engine | Overview |
|---|---|
| vLLM | High-throughput LLM serving engine built around PagedAttention. Handles concurrent requests and quantized weights (AWQ / GPTQ / FP8) and is widely used on server-class GPUs. |
| SGLang | LLM serving framework known for RadixAttention prefix caching and structured output. A common alternative to vLLM for serving. |
Models Available in AWQ
→ Scroll horizontally to see all columns
| Published | Model | Where to get it | Quantizations | Smallest VRAM tier | Article |
|---|---|---|---|---|---|
| 2026-11-18 | sensenova/SenseNova-Loop-9B-AWQ | This repository | — | — | SenseNova-Loop-9B-AWQ 512K Context Multimodal Model: GGUF Builds |
“Smallest VRAM tier" is the smallest tier in the requirements table of each article (for other builds, of those builds; for image, video and audio models, always the article’s own table, which counts every component). Leave headroom for context length.
Last updated 2026-11-18 (JST). The explanation at the top of this page was written with the help of AI from the primary sources it cites. The tables under “Our Coverage and Data" are assembled by code from our article log.