<\/p>\n
Values determined by this site’s code at collection time. Dates are JST.<\/em><\/p>\n<\/p>\nOverview<\/h2>\nOrcaRouter, an AI gateway operator, has released a quantized version of Qwen3.8-27B compressed to an average of 3.21 bits, named OrcaSAQ2 27B<\/strong> (orcarouter\/OrcaSAQ-2-27B<\/code>), on Hugging Face. The original model, which is 54GB in BF16, has been reduced to a total of 12.3GB across its distribution files. The stated goal of the publisher is to “run a 27B model on a single 16GB GPU<\/strong>,” and they present numerical comparisons directly against the BF16 version to show that the difference from the original model is small.<\/p>\nThe publisher calls this a “proprietary sensitivity-aware mixed-precision quantization system (SAQ2)” and has not disclosed the details of the method (calibration procedures, bit-width allocation per layer, or packing methods). However, looking at the distribution files’ configuration (quantization_config.json<\/code>), the quantization format is exl3<\/code> (ExLlamaV3 format, version 1.5.1). The publisher’s concurrently released execution kernel repository also describes its contents as “a combination of EXL3 trellis quantization (QTIP family) with mixed-precision allocation determined by search.” In other words, it is accurate to read this as not a new format, but rather a clever allocation of bit-widths per layer within the existing EXL3 format<\/strong>.<\/p>\nThe base Qwen3.8-27B model itself is covered in Qwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a>.<\/p>\nSpecifications<\/h2>\n\nBase Model: Qwen3.8-27B (Hybrid architecture of the Qwen3.5 family. Out of 64 layers, 48 are Gated DeltaNet and 16 are standard attention)<\/li>\nQuantization: EXL3 format, averaging 3.21 bits for the decoder part. Output layer (lm_head<\/code>) is 6-bit, embeddings are int8, and MTP heads are 4-bit<\/li>\nDistribution Files: Split into 4 safetensors files, total 12.27GB<\/li>\nContext Length: 262,144 tokens (design upper limit. The length actually usable on a 16GB GPU is described later)<\/li>\nSupported Features: Thinking mode, tool calling, speculative decoding via MTP (Multi-Token Prediction)<\/li>\nInput: Text only. While the base Qwen3.8-27B can read images and videos, this version does not include a visual encoder<\/li>\nLicense: Apache-2.0 (inherited from the base Qwen3.8-27B)<\/li>\n<\/ul>\nPerformance<\/h2>\nDeviation from the Original Model (BF16)<\/h3>\nThe publisher provides values comparing the distributed weights themselves using the same evaluation pipeline as the BF16 version, measured on 16,376 tokens of WikiText-2.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nBuild<\/th>\nSize<\/th>\nDecoder Bit-width<\/th>\nMean KLD \u2193<\/th>\nTop-1 Match Rate \u2191<\/th>\nPPL \u2193<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen3.8-27B BF16<\/td>\n54GB<\/td>\n16<\/td>\n\u2014<\/td>\n100%<\/td>\n5.6468<\/td>\n<\/tr>\n\nOrcaSAQ2 27B<\/td>\n12.3GB<\/td>\n3.21<\/td>\n0.031<\/td>\n93.2%<\/td>\n5.6482<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe meanings of the three metrics are as follows:<\/p>\n\nPPL (Perplexity)<\/strong>: Measures how well the model can predict the continuation of text without confusion. Lower is better. The degradation from BF16 is +0.02%, showing almost no difference<\/li>\nKLD (Kullback-Leibler Divergence)<\/strong>: Measures how much the probability distribution of the next token deviates from BF16. Closer to 0 means more faithful to the original model<\/li>\nTop-1 Match Rate<\/strong>: The proportion of times the “most likely next token chosen” matched the BF16 version<\/li>\n<\/ul>\nIt is important to read these three metrics side by side. Even though the PPL difference is only 0.02%, the Top-1 match rate is 93.2%, meaning roughly once every 15 tokens, a different token is chosen as the most likely candidate compared to BF16<\/strong>. Because PPL is an averaged value across the entire text, individual choices tend to cancel each other out and do not readily show up in the aggregate score. The publisher itself notes in the limitations section that “a 93.2% match rate means some token decisions differ from BF16” and “a +0.02% PPL does not guarantee identical performance on downstream tasks.” Rather than summarizing it as “virtually lossless,” it is more accurate to understand it as “average predictions barely change, but individual decisions change at a certain rate<\/strong>.” Note that comparisons measuring other quantized versions (like GGUF or other EXL3 variants) under the same conditions are not provided, so it is impossible to judge from this data alone whether a KLD of 0.031 is exceptional for the 3-bit range.<\/p>\nAgent Benchmarks (Publisher’s Self-Report)<\/h3>\nThe model card presents scores of 70.0 on SWE-bench Verified and 58.4 on Terminal-Bench 2.1 in a table alongside published values for Claude Sonnet 4.6, Gemini 3, GPT-5.4, and others. However, this article does not treat these as rankings against other models<\/strong> for the following reasons:<\/p>\n\nThe publisher itself clarifies that the table is “not a controlled comparison, but published reference values.” Agent benchmark scores vary significantly depending on the execution environment (harness), thinking budget, and time limits used<\/li>\nThe specific execution environment and settings used to measure the OrcaSAQ2 scores are not documented<\/li>\nThe most crucial piece of information\u2014”the score of the BF16 version measured under the exact same conditions<\/strong>“\u2014is missing. Therefore, it is impossible to tell from this table how much performance was lost due to quantization<\/li>\n<\/ul>\nThe only meaningful evaluation for the quantized version is the “deviation from BF16” discussed in the previous section. It is best not to read more into the agent benchmark scores than simply that the publisher reported them.<\/p>\nSpeed on a 16GB GPU<\/h3>\nValues measured with vLLM while restricting GPU memory to 15.7GiB.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nSetting<\/th>\n1 Stream<\/th>\n8 Streams<\/th>\n16 Streams<\/th>\nKV Cache Capacity<\/th>\n<\/tr>\n<\/thead>\n\n\nWithout MTP<\/td>\n65.3 tok\/s<\/td>\n332 tok\/s<\/td>\n333 tok\/s<\/td>\n29,354 tokens<\/td>\n<\/tr>\n\nWith MTP<\/td>\n90.1 tok\/s<\/td>\n220 tok\/s<\/td>\n219 tok\/s<\/td>\n14,563 tokens<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nUsing MTP (speculative decoding) makes generation about 38% faster for single-user use. In exchange, because the draft head takes KV cache from the same memory pool, the available context length is cut roughly in half, and it actually becomes slower when handling many concurrent requests<\/strong>. The choice comes down to using MTP enabled for single-user interactive use, and disabled for batching multiple requests.<\/p>\nThe execution kernel repository contains a table with the same intent, but the numbers differ (with a 15.5GiB limit, 1 stream with MTP is 111.2 tok\/s, and KV cache without MTP is 59,753 tokens). This is likely due to differences in settings or versions, but both are measurements by the publisher, and independent third-party replication is not yet available.<\/p>\nStrengths and Use Cases<\/h2>\nThe target use case envisioned by the publisher is agents performing long procedures involving coding, terminal operations, and browser navigation. The publisher’s concern is that even small errors from quantization can cause a single tool call to change, cascading to alter all subsequent states\u2014which is why tasks with long procedures are more susceptible to quantization effects, and why they detailed the “deviation from BF16” so thoroughly.<\/p>\nFrom a local user’s perspective, this model is suited for those who want to set up a 27B-class model as a server using a 16GB GPU (such as an RTX 5080 or 4080)<\/strong>. Since it runs as an OpenAI-compatible API, it can be integrated with coding agents or custom tools.<\/p>\nThere are also many points to keep in mind:<\/p>\n\nYou cannot use a 262K context on 16GB.<\/strong> Because the weights consume about 12.3GB, only about 3 to 4GB remains. The publisher also recommends “starting with a context around 32K on a 16GB GPU and adjusting according to the use case”<\/li>\nImages cannot be read.<\/strong> The visual capabilities of the base Qwen3.8-27B have been stripped out in this version<\/li>\nThe quantization method is private.<\/strong> You cannot apply the exact same procedure to other models yourself<\/li>\nThe publisher is a company operating an AI gateway, and many models released on Hugging Face are uncensored versions of existing models. This OrcaSAQ2 27B is released as a direct quantization of Qwen3.8-27B rather than an uncensored version<\/li>\n<\/ul>\nHow It Differs from Similar Models<\/h2>\n\nDifference from general GGUF quantized versions (Q3_K_M, IQ3 families, etc.)<\/strong>: While the file sizes are similar, the execution engines are completely different. While GGUF runs out of the box in llama.cpp, Ollama, and LM Studio, this version does not run in them (see the next section). Its strengths lie in providing numerical measurements of deviation from BF16 and the ability to handle multiple concurrent requests using vLLM<\/li>\nDifference from other EXL3 quantized versions<\/strong>: The EXL3 format itself is the standard format for ExLlamaV3, and other versions around 3 bits could potentially be created. The differences in OrcaSAQ2 lie in searching for layer-wise bit allocation distributions and providing a plugin to run it with vLLM. However, numerical comparisons against other EXL3 versions under identical conditions are not provided<\/li>\nDifference from the base Qwen3.8-27B (BF16)<\/strong>: The required memory drops to under a quarter, but image input capability is lost, and token-level agreement is limited to 93.2%. The choice is between the original model if accuracy is the top priority, or this version if fitting into 16GB is the priority<\/li>\n<\/ul>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 27.8B parameters (taken from the base model Qwen\/Qwen3.8-27B)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\n16GB (RTX 5060 Ti 16GB \/ 4060 Ti 16GB, etc.)<\/td>\nEXL3<\/td>\n11.4GB<\/td>\n13.7GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nNot usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\nThe publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\nLicense \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
OrcaRouter, an AI gateway operator, has released a quantized version of Qwen3.8-27B compressed to an average of 3.21 bits, named OrcaSAQ2 27B<\/strong> (orcarouter\/OrcaSAQ-2-27B<\/code>), on Hugging Face. The original model, which is 54GB in BF16, has been reduced to a total of 12.3GB across its distribution files. The stated goal of the publisher is to “run a 27B model on a single 16GB GPU<\/strong>,” and they present numerical comparisons directly against the BF16 version to show that the difference from the original model is small.<\/p>\nThe publisher calls this a “proprietary sensitivity-aware mixed-precision quantization system (SAQ2)” and has not disclosed the details of the method (calibration procedures, bit-width allocation per layer, or packing methods). However, looking at the distribution files’ configuration (quantization_config.json<\/code>), the quantization format is exl3<\/code> (ExLlamaV3 format, version 1.5.1). The publisher’s concurrently released execution kernel repository also describes its contents as “a combination of EXL3 trellis quantization (QTIP family) with mixed-precision allocation determined by search.” In other words, it is accurate to read this as not a new format, but rather a clever allocation of bit-widths per layer within the existing EXL3 format<\/strong>.<\/p>\nThe base Qwen3.8-27B model itself is covered in Qwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a>.<\/p>\nSpecifications<\/h2>\n\nBase Model: Qwen3.8-27B (Hybrid architecture of the Qwen3.5 family. Out of 64 layers, 48 are Gated DeltaNet and 16 are standard attention)<\/li>\nQuantization: EXL3 format, averaging 3.21 bits for the decoder part. Output layer (lm_head<\/code>) is 6-bit, embeddings are int8, and MTP heads are 4-bit<\/li>\nDistribution Files: Split into 4 safetensors files, total 12.27GB<\/li>\nContext Length: 262,144 tokens (design upper limit. The length actually usable on a 16GB GPU is described later)<\/li>\nSupported Features: Thinking mode, tool calling, speculative decoding via MTP (Multi-Token Prediction)<\/li>\nInput: Text only. While the base Qwen3.8-27B can read images and videos, this version does not include a visual encoder<\/li>\nLicense: Apache-2.0 (inherited from the base Qwen3.8-27B)<\/li>\n<\/ul>\nPerformance<\/h2>\nDeviation from the Original Model (BF16)<\/h3>\nThe publisher provides values comparing the distributed weights themselves using the same evaluation pipeline as the BF16 version, measured on 16,376 tokens of WikiText-2.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nBuild<\/th>\nSize<\/th>\nDecoder Bit-width<\/th>\nMean KLD \u2193<\/th>\nTop-1 Match Rate \u2191<\/th>\nPPL \u2193<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen3.8-27B BF16<\/td>\n54GB<\/td>\n16<\/td>\n\u2014<\/td>\n100%<\/td>\n5.6468<\/td>\n<\/tr>\n\nOrcaSAQ2 27B<\/td>\n12.3GB<\/td>\n3.21<\/td>\n0.031<\/td>\n93.2%<\/td>\n5.6482<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe meanings of the three metrics are as follows:<\/p>\n\nPPL (Perplexity)<\/strong>: Measures how well the model can predict the continuation of text without confusion. Lower is better. The degradation from BF16 is +0.02%, showing almost no difference<\/li>\nKLD (Kullback-Leibler Divergence)<\/strong>: Measures how much the probability distribution of the next token deviates from BF16. Closer to 0 means more faithful to the original model<\/li>\nTop-1 Match Rate<\/strong>: The proportion of times the “most likely next token chosen” matched the BF16 version<\/li>\n<\/ul>\nIt is important to read these three metrics side by side. Even though the PPL difference is only 0.02%, the Top-1 match rate is 93.2%, meaning roughly once every 15 tokens, a different token is chosen as the most likely candidate compared to BF16<\/strong>. Because PPL is an averaged value across the entire text, individual choices tend to cancel each other out and do not readily show up in the aggregate score. The publisher itself notes in the limitations section that “a 93.2% match rate means some token decisions differ from BF16” and “a +0.02% PPL does not guarantee identical performance on downstream tasks.” Rather than summarizing it as “virtually lossless,” it is more accurate to understand it as “average predictions barely change, but individual decisions change at a certain rate<\/strong>.” Note that comparisons measuring other quantized versions (like GGUF or other EXL3 variants) under the same conditions are not provided, so it is impossible to judge from this data alone whether a KLD of 0.031 is exceptional for the 3-bit range.<\/p>\nAgent Benchmarks (Publisher’s Self-Report)<\/h3>\nThe model card presents scores of 70.0 on SWE-bench Verified and 58.4 on Terminal-Bench 2.1 in a table alongside published values for Claude Sonnet 4.6, Gemini 3, GPT-5.4, and others. However, this article does not treat these as rankings against other models<\/strong> for the following reasons:<\/p>\n\nThe publisher itself clarifies that the table is “not a controlled comparison, but published reference values.” Agent benchmark scores vary significantly depending on the execution environment (harness), thinking budget, and time limits used<\/li>\nThe specific execution environment and settings used to measure the OrcaSAQ2 scores are not documented<\/li>\nThe most crucial piece of information\u2014”the score of the BF16 version measured under the exact same conditions<\/strong>“\u2014is missing. Therefore, it is impossible to tell from this table how much performance was lost due to quantization<\/li>\n<\/ul>\nThe only meaningful evaluation for the quantized version is the “deviation from BF16” discussed in the previous section. It is best not to read more into the agent benchmark scores than simply that the publisher reported them.<\/p>\nSpeed on a 16GB GPU<\/h3>\nValues measured with vLLM while restricting GPU memory to 15.7GiB.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nSetting<\/th>\n1 Stream<\/th>\n8 Streams<\/th>\n16 Streams<\/th>\nKV Cache Capacity<\/th>\n<\/tr>\n<\/thead>\n\n\nWithout MTP<\/td>\n65.3 tok\/s<\/td>\n332 tok\/s<\/td>\n333 tok\/s<\/td>\n29,354 tokens<\/td>\n<\/tr>\n\nWith MTP<\/td>\n90.1 tok\/s<\/td>\n220 tok\/s<\/td>\n219 tok\/s<\/td>\n14,563 tokens<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nUsing MTP (speculative decoding) makes generation about 38% faster for single-user use. In exchange, because the draft head takes KV cache from the same memory pool, the available context length is cut roughly in half, and it actually becomes slower when handling many concurrent requests<\/strong>. The choice comes down to using MTP enabled for single-user interactive use, and disabled for batching multiple requests.<\/p>\nThe execution kernel repository contains a table with the same intent, but the numbers differ (with a 15.5GiB limit, 1 stream with MTP is 111.2 tok\/s, and KV cache without MTP is 59,753 tokens). This is likely due to differences in settings or versions, but both are measurements by the publisher, and independent third-party replication is not yet available.<\/p>\nStrengths and Use Cases<\/h2>\nThe target use case envisioned by the publisher is agents performing long procedures involving coding, terminal operations, and browser navigation. The publisher’s concern is that even small errors from quantization can cause a single tool call to change, cascading to alter all subsequent states\u2014which is why tasks with long procedures are more susceptible to quantization effects, and why they detailed the “deviation from BF16” so thoroughly.<\/p>\nFrom a local user’s perspective, this model is suited for those who want to set up a 27B-class model as a server using a 16GB GPU (such as an RTX 5080 or 4080)<\/strong>. Since it runs as an OpenAI-compatible API, it can be integrated with coding agents or custom tools.<\/p>\nThere are also many points to keep in mind:<\/p>\n\nYou cannot use a 262K context on 16GB.<\/strong> Because the weights consume about 12.3GB, only about 3 to 4GB remains. The publisher also recommends “starting with a context around 32K on a 16GB GPU and adjusting according to the use case”<\/li>\nImages cannot be read.<\/strong> The visual capabilities of the base Qwen3.8-27B have been stripped out in this version<\/li>\nThe quantization method is private.<\/strong> You cannot apply the exact same procedure to other models yourself<\/li>\nThe publisher is a company operating an AI gateway, and many models released on Hugging Face are uncensored versions of existing models. This OrcaSAQ2 27B is released as a direct quantization of Qwen3.8-27B rather than an uncensored version<\/li>\n<\/ul>\nHow It Differs from Similar Models<\/h2>\n\nDifference from general GGUF quantized versions (Q3_K_M, IQ3 families, etc.)<\/strong>: While the file sizes are similar, the execution engines are completely different. While GGUF runs out of the box in llama.cpp, Ollama, and LM Studio, this version does not run in them (see the next section). Its strengths lie in providing numerical measurements of deviation from BF16 and the ability to handle multiple concurrent requests using vLLM<\/li>\nDifference from other EXL3 quantized versions<\/strong>: The EXL3 format itself is the standard format for ExLlamaV3, and other versions around 3 bits could potentially be created. The differences in OrcaSAQ2 lie in searching for layer-wise bit allocation distributions and providing a plugin to run it with vLLM. However, numerical comparisons against other EXL3 versions under identical conditions are not provided<\/li>\nDifference from the base Qwen3.8-27B (BF16)<\/strong>: The required memory drops to under a quarter, but image input capability is lost, and token-level agreement is limited to 93.2%. The choice is between the original model if accuracy is the top priority, or this version if fitting into 16GB is the priority<\/li>\n<\/ul>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 27.8B parameters (taken from the base model Qwen\/Qwen3.8-27B)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\n16GB (RTX 5060 Ti 16GB \/ 4060 Ti 16GB, etc.)<\/td>\nEXL3<\/td>\n11.4GB<\/td>\n13.7GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nNot usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\nThe publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\nLicense \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
orcarouter\/OrcaSAQ-2-27B<\/code>), on Hugging Face. The original model, which is 54GB in BF16, has been reduced to a total of 12.3GB across its distribution files. The stated goal of the publisher is to “run a 27B model on a single 16GB GPU<\/strong>,” and they present numerical comparisons directly against the BF16 version to show that the difference from the original model is small.<\/p>\nThe publisher calls this a “proprietary sensitivity-aware mixed-precision quantization system (SAQ2)” and has not disclosed the details of the method (calibration procedures, bit-width allocation per layer, or packing methods). However, looking at the distribution files’ configuration (quantization_config.json<\/code>), the quantization format is exl3<\/code> (ExLlamaV3 format, version 1.5.1). The publisher’s concurrently released execution kernel repository also describes its contents as “a combination of EXL3 trellis quantization (QTIP family) with mixed-precision allocation determined by search.” In other words, it is accurate to read this as not a new format, but rather a clever allocation of bit-widths per layer within the existing EXL3 format<\/strong>.<\/p>\nThe base Qwen3.8-27B model itself is covered in Qwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a>.<\/p>\nSpecifications<\/h2>\n\nBase Model: Qwen3.8-27B (Hybrid architecture of the Qwen3.5 family. Out of 64 layers, 48 are Gated DeltaNet and 16 are standard attention)<\/li>\nQuantization: EXL3 format, averaging 3.21 bits for the decoder part. Output layer (lm_head<\/code>) is 6-bit, embeddings are int8, and MTP heads are 4-bit<\/li>\nDistribution Files: Split into 4 safetensors files, total 12.27GB<\/li>\nContext Length: 262,144 tokens (design upper limit. The length actually usable on a 16GB GPU is described later)<\/li>\nSupported Features: Thinking mode, tool calling, speculative decoding via MTP (Multi-Token Prediction)<\/li>\nInput: Text only. While the base Qwen3.8-27B can read images and videos, this version does not include a visual encoder<\/li>\nLicense: Apache-2.0 (inherited from the base Qwen3.8-27B)<\/li>\n<\/ul>\nPerformance<\/h2>\nDeviation from the Original Model (BF16)<\/h3>\nThe publisher provides values comparing the distributed weights themselves using the same evaluation pipeline as the BF16 version, measured on 16,376 tokens of WikiText-2.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nBuild<\/th>\nSize<\/th>\nDecoder Bit-width<\/th>\nMean KLD \u2193<\/th>\nTop-1 Match Rate \u2191<\/th>\nPPL \u2193<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen3.8-27B BF16<\/td>\n54GB<\/td>\n16<\/td>\n\u2014<\/td>\n100%<\/td>\n5.6468<\/td>\n<\/tr>\n\nOrcaSAQ2 27B<\/td>\n12.3GB<\/td>\n3.21<\/td>\n0.031<\/td>\n93.2%<\/td>\n5.6482<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe meanings of the three metrics are as follows:<\/p>\n\nPPL (Perplexity)<\/strong>: Measures how well the model can predict the continuation of text without confusion. Lower is better. The degradation from BF16 is +0.02%, showing almost no difference<\/li>\nKLD (Kullback-Leibler Divergence)<\/strong>: Measures how much the probability distribution of the next token deviates from BF16. Closer to 0 means more faithful to the original model<\/li>\nTop-1 Match Rate<\/strong>: The proportion of times the “most likely next token chosen” matched the BF16 version<\/li>\n<\/ul>\nIt is important to read these three metrics side by side. Even though the PPL difference is only 0.02%, the Top-1 match rate is 93.2%, meaning roughly once every 15 tokens, a different token is chosen as the most likely candidate compared to BF16<\/strong>. Because PPL is an averaged value across the entire text, individual choices tend to cancel each other out and do not readily show up in the aggregate score. The publisher itself notes in the limitations section that “a 93.2% match rate means some token decisions differ from BF16” and “a +0.02% PPL does not guarantee identical performance on downstream tasks.” Rather than summarizing it as “virtually lossless,” it is more accurate to understand it as “average predictions barely change, but individual decisions change at a certain rate<\/strong>.” Note that comparisons measuring other quantized versions (like GGUF or other EXL3 variants) under the same conditions are not provided, so it is impossible to judge from this data alone whether a KLD of 0.031 is exceptional for the 3-bit range.<\/p>\nAgent Benchmarks (Publisher’s Self-Report)<\/h3>\nThe model card presents scores of 70.0 on SWE-bench Verified and 58.4 on Terminal-Bench 2.1 in a table alongside published values for Claude Sonnet 4.6, Gemini 3, GPT-5.4, and others. However, this article does not treat these as rankings against other models<\/strong> for the following reasons:<\/p>\n\nThe publisher itself clarifies that the table is “not a controlled comparison, but published reference values.” Agent benchmark scores vary significantly depending on the execution environment (harness), thinking budget, and time limits used<\/li>\nThe specific execution environment and settings used to measure the OrcaSAQ2 scores are not documented<\/li>\nThe most crucial piece of information\u2014”the score of the BF16 version measured under the exact same conditions<\/strong>“\u2014is missing. Therefore, it is impossible to tell from this table how much performance was lost due to quantization<\/li>\n<\/ul>\nThe only meaningful evaluation for the quantized version is the “deviation from BF16” discussed in the previous section. It is best not to read more into the agent benchmark scores than simply that the publisher reported them.<\/p>\nSpeed on a 16GB GPU<\/h3>\nValues measured with vLLM while restricting GPU memory to 15.7GiB.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nSetting<\/th>\n1 Stream<\/th>\n8 Streams<\/th>\n16 Streams<\/th>\nKV Cache Capacity<\/th>\n<\/tr>\n<\/thead>\n\n\nWithout MTP<\/td>\n65.3 tok\/s<\/td>\n332 tok\/s<\/td>\n333 tok\/s<\/td>\n29,354 tokens<\/td>\n<\/tr>\n\nWith MTP<\/td>\n90.1 tok\/s<\/td>\n220 tok\/s<\/td>\n219 tok\/s<\/td>\n14,563 tokens<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nUsing MTP (speculative decoding) makes generation about 38% faster for single-user use. In exchange, because the draft head takes KV cache from the same memory pool, the available context length is cut roughly in half, and it actually becomes slower when handling many concurrent requests<\/strong>. The choice comes down to using MTP enabled for single-user interactive use, and disabled for batching multiple requests.<\/p>\nThe execution kernel repository contains a table with the same intent, but the numbers differ (with a 15.5GiB limit, 1 stream with MTP is 111.2 tok\/s, and KV cache without MTP is 59,753 tokens). This is likely due to differences in settings or versions, but both are measurements by the publisher, and independent third-party replication is not yet available.<\/p>\nStrengths and Use Cases<\/h2>\nThe target use case envisioned by the publisher is agents performing long procedures involving coding, terminal operations, and browser navigation. The publisher’s concern is that even small errors from quantization can cause a single tool call to change, cascading to alter all subsequent states\u2014which is why tasks with long procedures are more susceptible to quantization effects, and why they detailed the “deviation from BF16” so thoroughly.<\/p>\nFrom a local user’s perspective, this model is suited for those who want to set up a 27B-class model as a server using a 16GB GPU (such as an RTX 5080 or 4080)<\/strong>. Since it runs as an OpenAI-compatible API, it can be integrated with coding agents or custom tools.<\/p>\nThere are also many points to keep in mind:<\/p>\n\nYou cannot use a 262K context on 16GB.<\/strong> Because the weights consume about 12.3GB, only about 3 to 4GB remains. The publisher also recommends “starting with a context around 32K on a 16GB GPU and adjusting according to the use case”<\/li>\nImages cannot be read.<\/strong> The visual capabilities of the base Qwen3.8-27B have been stripped out in this version<\/li>\nThe quantization method is private.<\/strong> You cannot apply the exact same procedure to other models yourself<\/li>\nThe publisher is a company operating an AI gateway, and many models released on Hugging Face are uncensored versions of existing models. This OrcaSAQ2 27B is released as a direct quantization of Qwen3.8-27B rather than an uncensored version<\/li>\n<\/ul>\nHow It Differs from Similar Models<\/h2>\n\nDifference from general GGUF quantized versions (Q3_K_M, IQ3 families, etc.)<\/strong>: While the file sizes are similar, the execution engines are completely different. While GGUF runs out of the box in llama.cpp, Ollama, and LM Studio, this version does not run in them (see the next section). Its strengths lie in providing numerical measurements of deviation from BF16 and the ability to handle multiple concurrent requests using vLLM<\/li>\nDifference from other EXL3 quantized versions<\/strong>: The EXL3 format itself is the standard format for ExLlamaV3, and other versions around 3 bits could potentially be created. The differences in OrcaSAQ2 lie in searching for layer-wise bit allocation distributions and providing a plugin to run it with vLLM. However, numerical comparisons against other EXL3 versions under identical conditions are not provided<\/li>\nDifference from the base Qwen3.8-27B (BF16)<\/strong>: The required memory drops to under a quarter, but image input capability is lost, and token-level agreement is limited to 93.2%. The choice is between the original model if accuracy is the top priority, or this version if fitting into 16GB is the priority<\/li>\n<\/ul>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 27.8B parameters (taken from the base model Qwen\/Qwen3.8-27B)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\n16GB (RTX 5060 Ti 16GB \/ 4060 Ti 16GB, etc.)<\/td>\nEXL3<\/td>\n11.4GB<\/td>\n13.7GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nNot usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\nThe publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\nLicense \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The publisher calls this a “proprietary sensitivity-aware mixed-precision quantization system (SAQ2)” and has not disclosed the details of the method (calibration procedures, bit-width allocation per layer, or packing methods). However, looking at the distribution files’ configuration (quantization_config.json<\/code>), the quantization format is exl3<\/code> (ExLlamaV3 format, version 1.5.1). The publisher’s concurrently released execution kernel repository also describes its contents as “a combination of EXL3 trellis quantization (QTIP family) with mixed-precision allocation determined by search.” In other words, it is accurate to read this as not a new format, but rather a clever allocation of bit-widths per layer within the existing EXL3 format<\/strong>.<\/p>\nThe base Qwen3.8-27B model itself is covered in Qwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a>.<\/p>\nSpecifications<\/h2>\n\nBase Model: Qwen3.8-27B (Hybrid architecture of the Qwen3.5 family. Out of 64 layers, 48 are Gated DeltaNet and 16 are standard attention)<\/li>\nQuantization: EXL3 format, averaging 3.21 bits for the decoder part. Output layer (lm_head<\/code>) is 6-bit, embeddings are int8, and MTP heads are 4-bit<\/li>\nDistribution Files: Split into 4 safetensors files, total 12.27GB<\/li>\nContext Length: 262,144 tokens (design upper limit. The length actually usable on a 16GB GPU is described later)<\/li>\nSupported Features: Thinking mode, tool calling, speculative decoding via MTP (Multi-Token Prediction)<\/li>\nInput: Text only. While the base Qwen3.8-27B can read images and videos, this version does not include a visual encoder<\/li>\nLicense: Apache-2.0 (inherited from the base Qwen3.8-27B)<\/li>\n<\/ul>\nPerformance<\/h2>\nDeviation from the Original Model (BF16)<\/h3>\nThe publisher provides values comparing the distributed weights themselves using the same evaluation pipeline as the BF16 version, measured on 16,376 tokens of WikiText-2.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nBuild<\/th>\nSize<\/th>\nDecoder Bit-width<\/th>\nMean KLD \u2193<\/th>\nTop-1 Match Rate \u2191<\/th>\nPPL \u2193<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen3.8-27B BF16<\/td>\n54GB<\/td>\n16<\/td>\n\u2014<\/td>\n100%<\/td>\n5.6468<\/td>\n<\/tr>\n\nOrcaSAQ2 27B<\/td>\n12.3GB<\/td>\n3.21<\/td>\n0.031<\/td>\n93.2%<\/td>\n5.6482<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe meanings of the three metrics are as follows:<\/p>\n\nPPL (Perplexity)<\/strong>: Measures how well the model can predict the continuation of text without confusion. Lower is better. The degradation from BF16 is +0.02%, showing almost no difference<\/li>\nKLD (Kullback-Leibler Divergence)<\/strong>: Measures how much the probability distribution of the next token deviates from BF16. Closer to 0 means more faithful to the original model<\/li>\nTop-1 Match Rate<\/strong>: The proportion of times the “most likely next token chosen” matched the BF16 version<\/li>\n<\/ul>\nIt is important to read these three metrics side by side. Even though the PPL difference is only 0.02%, the Top-1 match rate is 93.2%, meaning roughly once every 15 tokens, a different token is chosen as the most likely candidate compared to BF16<\/strong>. Because PPL is an averaged value across the entire text, individual choices tend to cancel each other out and do not readily show up in the aggregate score. The publisher itself notes in the limitations section that “a 93.2% match rate means some token decisions differ from BF16” and “a +0.02% PPL does not guarantee identical performance on downstream tasks.” Rather than summarizing it as “virtually lossless,” it is more accurate to understand it as “average predictions barely change, but individual decisions change at a certain rate<\/strong>.” Note that comparisons measuring other quantized versions (like GGUF or other EXL3 variants) under the same conditions are not provided, so it is impossible to judge from this data alone whether a KLD of 0.031 is exceptional for the 3-bit range.<\/p>\nAgent Benchmarks (Publisher’s Self-Report)<\/h3>\nThe model card presents scores of 70.0 on SWE-bench Verified and 58.4 on Terminal-Bench 2.1 in a table alongside published values for Claude Sonnet 4.6, Gemini 3, GPT-5.4, and others. However, this article does not treat these as rankings against other models<\/strong> for the following reasons:<\/p>\n\nThe publisher itself clarifies that the table is “not a controlled comparison, but published reference values.” Agent benchmark scores vary significantly depending on the execution environment (harness), thinking budget, and time limits used<\/li>\nThe specific execution environment and settings used to measure the OrcaSAQ2 scores are not documented<\/li>\nThe most crucial piece of information\u2014”the score of the BF16 version measured under the exact same conditions<\/strong>“\u2014is missing. Therefore, it is impossible to tell from this table how much performance was lost due to quantization<\/li>\n<\/ul>\nThe only meaningful evaluation for the quantized version is the “deviation from BF16” discussed in the previous section. It is best not to read more into the agent benchmark scores than simply that the publisher reported them.<\/p>\nSpeed on a 16GB GPU<\/h3>\nValues measured with vLLM while restricting GPU memory to 15.7GiB.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nSetting<\/th>\n1 Stream<\/th>\n8 Streams<\/th>\n16 Streams<\/th>\nKV Cache Capacity<\/th>\n<\/tr>\n<\/thead>\n\n\nWithout MTP<\/td>\n65.3 tok\/s<\/td>\n332 tok\/s<\/td>\n333 tok\/s<\/td>\n29,354 tokens<\/td>\n<\/tr>\n\nWith MTP<\/td>\n90.1 tok\/s<\/td>\n220 tok\/s<\/td>\n219 tok\/s<\/td>\n14,563 tokens<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nUsing MTP (speculative decoding) makes generation about 38% faster for single-user use. In exchange, because the draft head takes KV cache from the same memory pool, the available context length is cut roughly in half, and it actually becomes slower when handling many concurrent requests<\/strong>. The choice comes down to using MTP enabled for single-user interactive use, and disabled for batching multiple requests.<\/p>\nThe execution kernel repository contains a table with the same intent, but the numbers differ (with a 15.5GiB limit, 1 stream with MTP is 111.2 tok\/s, and KV cache without MTP is 59,753 tokens). This is likely due to differences in settings or versions, but both are measurements by the publisher, and independent third-party replication is not yet available.<\/p>\nStrengths and Use Cases<\/h2>\nThe target use case envisioned by the publisher is agents performing long procedures involving coding, terminal operations, and browser navigation. The publisher’s concern is that even small errors from quantization can cause a single tool call to change, cascading to alter all subsequent states\u2014which is why tasks with long procedures are more susceptible to quantization effects, and why they detailed the “deviation from BF16” so thoroughly.<\/p>\nFrom a local user’s perspective, this model is suited for those who want to set up a 27B-class model as a server using a 16GB GPU (such as an RTX 5080 or 4080)<\/strong>. Since it runs as an OpenAI-compatible API, it can be integrated with coding agents or custom tools.<\/p>\nThere are also many points to keep in mind:<\/p>\n\nYou cannot use a 262K context on 16GB.<\/strong> Because the weights consume about 12.3GB, only about 3 to 4GB remains. The publisher also recommends “starting with a context around 32K on a 16GB GPU and adjusting according to the use case”<\/li>\nImages cannot be read.<\/strong> The visual capabilities of the base Qwen3.8-27B have been stripped out in this version<\/li>\nThe quantization method is private.<\/strong> You cannot apply the exact same procedure to other models yourself<\/li>\nThe publisher is a company operating an AI gateway, and many models released on Hugging Face are uncensored versions of existing models. This OrcaSAQ2 27B is released as a direct quantization of Qwen3.8-27B rather than an uncensored version<\/li>\n<\/ul>\nHow It Differs from Similar Models<\/h2>\n\nDifference from general GGUF quantized versions (Q3_K_M, IQ3 families, etc.)<\/strong>: While the file sizes are similar, the execution engines are completely different. While GGUF runs out of the box in llama.cpp, Ollama, and LM Studio, this version does not run in them (see the next section). Its strengths lie in providing numerical measurements of deviation from BF16 and the ability to handle multiple concurrent requests using vLLM<\/li>\nDifference from other EXL3 quantized versions<\/strong>: The EXL3 format itself is the standard format for ExLlamaV3, and other versions around 3 bits could potentially be created. The differences in OrcaSAQ2 lie in searching for layer-wise bit allocation distributions and providing a plugin to run it with vLLM. However, numerical comparisons against other EXL3 versions under identical conditions are not provided<\/li>\nDifference from the base Qwen3.8-27B (BF16)<\/strong>: The required memory drops to under a quarter, but image input capability is lost, and token-level agreement is limited to 93.2%. The choice is between the original model if accuracy is the top priority, or this version if fitting into 16GB is the priority<\/li>\n<\/ul>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 27.8B parameters (taken from the base model Qwen\/Qwen3.8-27B)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\n16GB (RTX 5060 Ti 16GB \/ 4060 Ti 16GB, etc.)<\/td>\nEXL3<\/td>\n11.4GB<\/td>\n13.7GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nNot usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\nThe publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\nLicense \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
quantization_config.json<\/code>), the quantization format is exl3<\/code> (ExLlamaV3 format, version 1.5.1). The publisher’s concurrently released execution kernel repository also describes its contents as “a combination of EXL3 trellis quantization (QTIP family) with mixed-precision allocation determined by search.” In other words, it is accurate to read this as not a new format, but rather a clever allocation of bit-widths per layer within the existing EXL3 format<\/strong>.<\/p>\nThe base Qwen3.8-27B model itself is covered in Qwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a>.<\/p>\nSpecifications<\/h2>\n\nBase Model: Qwen3.8-27B (Hybrid architecture of the Qwen3.5 family. Out of 64 layers, 48 are Gated DeltaNet and 16 are standard attention)<\/li>\nQuantization: EXL3 format, averaging 3.21 bits for the decoder part. Output layer (lm_head<\/code>) is 6-bit, embeddings are int8, and MTP heads are 4-bit<\/li>\nDistribution Files: Split into 4 safetensors files, total 12.27GB<\/li>\nContext Length: 262,144 tokens (design upper limit. The length actually usable on a 16GB GPU is described later)<\/li>\nSupported Features: Thinking mode, tool calling, speculative decoding via MTP (Multi-Token Prediction)<\/li>\nInput: Text only. While the base Qwen3.8-27B can read images and videos, this version does not include a visual encoder<\/li>\nLicense: Apache-2.0 (inherited from the base Qwen3.8-27B)<\/li>\n<\/ul>\nPerformance<\/h2>\nDeviation from the Original Model (BF16)<\/h3>\nThe publisher provides values comparing the distributed weights themselves using the same evaluation pipeline as the BF16 version, measured on 16,376 tokens of WikiText-2.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nBuild<\/th>\nSize<\/th>\nDecoder Bit-width<\/th>\nMean KLD \u2193<\/th>\nTop-1 Match Rate \u2191<\/th>\nPPL \u2193<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen3.8-27B BF16<\/td>\n54GB<\/td>\n16<\/td>\n\u2014<\/td>\n100%<\/td>\n5.6468<\/td>\n<\/tr>\n\nOrcaSAQ2 27B<\/td>\n12.3GB<\/td>\n3.21<\/td>\n0.031<\/td>\n93.2%<\/td>\n5.6482<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe meanings of the three metrics are as follows:<\/p>\n\nPPL (Perplexity)<\/strong>: Measures how well the model can predict the continuation of text without confusion. Lower is better. The degradation from BF16 is +0.02%, showing almost no difference<\/li>\nKLD (Kullback-Leibler Divergence)<\/strong>: Measures how much the probability distribution of the next token deviates from BF16. Closer to 0 means more faithful to the original model<\/li>\nTop-1 Match Rate<\/strong>: The proportion of times the “most likely next token chosen” matched the BF16 version<\/li>\n<\/ul>\nIt is important to read these three metrics side by side. Even though the PPL difference is only 0.02%, the Top-1 match rate is 93.2%, meaning roughly once every 15 tokens, a different token is chosen as the most likely candidate compared to BF16<\/strong>. Because PPL is an averaged value across the entire text, individual choices tend to cancel each other out and do not readily show up in the aggregate score. The publisher itself notes in the limitations section that “a 93.2% match rate means some token decisions differ from BF16” and “a +0.02% PPL does not guarantee identical performance on downstream tasks.” Rather than summarizing it as “virtually lossless,” it is more accurate to understand it as “average predictions barely change, but individual decisions change at a certain rate<\/strong>.” Note that comparisons measuring other quantized versions (like GGUF or other EXL3 variants) under the same conditions are not provided, so it is impossible to judge from this data alone whether a KLD of 0.031 is exceptional for the 3-bit range.<\/p>\nAgent Benchmarks (Publisher’s Self-Report)<\/h3>\nThe model card presents scores of 70.0 on SWE-bench Verified and 58.4 on Terminal-Bench 2.1 in a table alongside published values for Claude Sonnet 4.6, Gemini 3, GPT-5.4, and others. However, this article does not treat these as rankings against other models<\/strong> for the following reasons:<\/p>\n\nThe publisher itself clarifies that the table is “not a controlled comparison, but published reference values.” Agent benchmark scores vary significantly depending on the execution environment (harness), thinking budget, and time limits used<\/li>\nThe specific execution environment and settings used to measure the OrcaSAQ2 scores are not documented<\/li>\nThe most crucial piece of information\u2014”the score of the BF16 version measured under the exact same conditions<\/strong>“\u2014is missing. Therefore, it is impossible to tell from this table how much performance was lost due to quantization<\/li>\n<\/ul>\nThe only meaningful evaluation for the quantized version is the “deviation from BF16” discussed in the previous section. It is best not to read more into the agent benchmark scores than simply that the publisher reported them.<\/p>\nSpeed on a 16GB GPU<\/h3>\nValues measured with vLLM while restricting GPU memory to 15.7GiB.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nSetting<\/th>\n1 Stream<\/th>\n8 Streams<\/th>\n16 Streams<\/th>\nKV Cache Capacity<\/th>\n<\/tr>\n<\/thead>\n\n\nWithout MTP<\/td>\n65.3 tok\/s<\/td>\n332 tok\/s<\/td>\n333 tok\/s<\/td>\n29,354 tokens<\/td>\n<\/tr>\n\nWith MTP<\/td>\n90.1 tok\/s<\/td>\n220 tok\/s<\/td>\n219 tok\/s<\/td>\n14,563 tokens<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nUsing MTP (speculative decoding) makes generation about 38% faster for single-user use. In exchange, because the draft head takes KV cache from the same memory pool, the available context length is cut roughly in half, and it actually becomes slower when handling many concurrent requests<\/strong>. The choice comes down to using MTP enabled for single-user interactive use, and disabled for batching multiple requests.<\/p>\nThe execution kernel repository contains a table with the same intent, but the numbers differ (with a 15.5GiB limit, 1 stream with MTP is 111.2 tok\/s, and KV cache without MTP is 59,753 tokens). This is likely due to differences in settings or versions, but both are measurements by the publisher, and independent third-party replication is not yet available.<\/p>\nStrengths and Use Cases<\/h2>\nThe target use case envisioned by the publisher is agents performing long procedures involving coding, terminal operations, and browser navigation. The publisher’s concern is that even small errors from quantization can cause a single tool call to change, cascading to alter all subsequent states\u2014which is why tasks with long procedures are more susceptible to quantization effects, and why they detailed the “deviation from BF16” so thoroughly.<\/p>\nFrom a local user’s perspective, this model is suited for those who want to set up a 27B-class model as a server using a 16GB GPU (such as an RTX 5080 or 4080)<\/strong>. Since it runs as an OpenAI-compatible API, it can be integrated with coding agents or custom tools.<\/p>\nThere are also many points to keep in mind:<\/p>\n\nYou cannot use a 262K context on 16GB.<\/strong> Because the weights consume about 12.3GB, only about 3 to 4GB remains. The publisher also recommends “starting with a context around 32K on a 16GB GPU and adjusting according to the use case”<\/li>\nImages cannot be read.<\/strong> The visual capabilities of the base Qwen3.8-27B have been stripped out in this version<\/li>\nThe quantization method is private.<\/strong> You cannot apply the exact same procedure to other models yourself<\/li>\nThe publisher is a company operating an AI gateway, and many models released on Hugging Face are uncensored versions of existing models. This OrcaSAQ2 27B is released as a direct quantization of Qwen3.8-27B rather than an uncensored version<\/li>\n<\/ul>\nHow It Differs from Similar Models<\/h2>\n\nDifference from general GGUF quantized versions (Q3_K_M, IQ3 families, etc.)<\/strong>: While the file sizes are similar, the execution engines are completely different. While GGUF runs out of the box in llama.cpp, Ollama, and LM Studio, this version does not run in them (see the next section). Its strengths lie in providing numerical measurements of deviation from BF16 and the ability to handle multiple concurrent requests using vLLM<\/li>\nDifference from other EXL3 quantized versions<\/strong>: The EXL3 format itself is the standard format for ExLlamaV3, and other versions around 3 bits could potentially be created. The differences in OrcaSAQ2 lie in searching for layer-wise bit allocation distributions and providing a plugin to run it with vLLM. However, numerical comparisons against other EXL3 versions under identical conditions are not provided<\/li>\nDifference from the base Qwen3.8-27B (BF16)<\/strong>: The required memory drops to under a quarter, but image input capability is lost, and token-level agreement is limited to 93.2%. The choice is between the original model if accuracy is the top priority, or this version if fitting into 16GB is the priority<\/li>\n<\/ul>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 27.8B parameters (taken from the base model Qwen\/Qwen3.8-27B)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\n16GB (RTX 5060 Ti 16GB \/ 4060 Ti 16GB, etc.)<\/td>\nEXL3<\/td>\n11.4GB<\/td>\n13.7GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nNot usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\nThe publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\nLicense \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
exl3<\/code> (ExLlamaV3 format, version 1.5.1). The publisher’s concurrently released execution kernel repository also describes its contents as “a combination of EXL3 trellis quantization (QTIP family) with mixed-precision allocation determined by search.” In other words, it is accurate to read this as not a new format, but rather a clever allocation of bit-widths per layer within the existing EXL3 format<\/strong>.<\/p>\nThe base Qwen3.8-27B model itself is covered in Qwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a>.<\/p>\nSpecifications<\/h2>\n\nBase Model: Qwen3.8-27B (Hybrid architecture of the Qwen3.5 family. Out of 64 layers, 48 are Gated DeltaNet and 16 are standard attention)<\/li>\nQuantization: EXL3 format, averaging 3.21 bits for the decoder part. Output layer (lm_head<\/code>) is 6-bit, embeddings are int8, and MTP heads are 4-bit<\/li>\nDistribution Files: Split into 4 safetensors files, total 12.27GB<\/li>\nContext Length: 262,144 tokens (design upper limit. The length actually usable on a 16GB GPU is described later)<\/li>\nSupported Features: Thinking mode, tool calling, speculative decoding via MTP (Multi-Token Prediction)<\/li>\nInput: Text only. While the base Qwen3.8-27B can read images and videos, this version does not include a visual encoder<\/li>\nLicense: Apache-2.0 (inherited from the base Qwen3.8-27B)<\/li>\n<\/ul>\nPerformance<\/h2>\nDeviation from the Original Model (BF16)<\/h3>\nThe publisher provides values comparing the distributed weights themselves using the same evaluation pipeline as the BF16 version, measured on 16,376 tokens of WikiText-2.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nBuild<\/th>\nSize<\/th>\nDecoder Bit-width<\/th>\nMean KLD \u2193<\/th>\nTop-1 Match Rate \u2191<\/th>\nPPL \u2193<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen3.8-27B BF16<\/td>\n54GB<\/td>\n16<\/td>\n\u2014<\/td>\n100%<\/td>\n5.6468<\/td>\n<\/tr>\n\nOrcaSAQ2 27B<\/td>\n12.3GB<\/td>\n3.21<\/td>\n0.031<\/td>\n93.2%<\/td>\n5.6482<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe meanings of the three metrics are as follows:<\/p>\n\nPPL (Perplexity)<\/strong>: Measures how well the model can predict the continuation of text without confusion. Lower is better. The degradation from BF16 is +0.02%, showing almost no difference<\/li>\nKLD (Kullback-Leibler Divergence)<\/strong>: Measures how much the probability distribution of the next token deviates from BF16. Closer to 0 means more faithful to the original model<\/li>\nTop-1 Match Rate<\/strong>: The proportion of times the “most likely next token chosen” matched the BF16 version<\/li>\n<\/ul>\nIt is important to read these three metrics side by side. Even though the PPL difference is only 0.02%, the Top-1 match rate is 93.2%, meaning roughly once every 15 tokens, a different token is chosen as the most likely candidate compared to BF16<\/strong>. Because PPL is an averaged value across the entire text, individual choices tend to cancel each other out and do not readily show up in the aggregate score. The publisher itself notes in the limitations section that “a 93.2% match rate means some token decisions differ from BF16” and “a +0.02% PPL does not guarantee identical performance on downstream tasks.” Rather than summarizing it as “virtually lossless,” it is more accurate to understand it as “average predictions barely change, but individual decisions change at a certain rate<\/strong>.” Note that comparisons measuring other quantized versions (like GGUF or other EXL3 variants) under the same conditions are not provided, so it is impossible to judge from this data alone whether a KLD of 0.031 is exceptional for the 3-bit range.<\/p>\nAgent Benchmarks (Publisher’s Self-Report)<\/h3>\nThe model card presents scores of 70.0 on SWE-bench Verified and 58.4 on Terminal-Bench 2.1 in a table alongside published values for Claude Sonnet 4.6, Gemini 3, GPT-5.4, and others. However, this article does not treat these as rankings against other models<\/strong> for the following reasons:<\/p>\n\nThe publisher itself clarifies that the table is “not a controlled comparison, but published reference values.” Agent benchmark scores vary significantly depending on the execution environment (harness), thinking budget, and time limits used<\/li>\nThe specific execution environment and settings used to measure the OrcaSAQ2 scores are not documented<\/li>\nThe most crucial piece of information\u2014”the score of the BF16 version measured under the exact same conditions<\/strong>“\u2014is missing. Therefore, it is impossible to tell from this table how much performance was lost due to quantization<\/li>\n<\/ul>\nThe only meaningful evaluation for the quantized version is the “deviation from BF16” discussed in the previous section. It is best not to read more into the agent benchmark scores than simply that the publisher reported them.<\/p>\nSpeed on a 16GB GPU<\/h3>\nValues measured with vLLM while restricting GPU memory to 15.7GiB.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nSetting<\/th>\n1 Stream<\/th>\n8 Streams<\/th>\n16 Streams<\/th>\nKV Cache Capacity<\/th>\n<\/tr>\n<\/thead>\n\n\nWithout MTP<\/td>\n65.3 tok\/s<\/td>\n332 tok\/s<\/td>\n333 tok\/s<\/td>\n29,354 tokens<\/td>\n<\/tr>\n\nWith MTP<\/td>\n90.1 tok\/s<\/td>\n220 tok\/s<\/td>\n219 tok\/s<\/td>\n14,563 tokens<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nUsing MTP (speculative decoding) makes generation about 38% faster for single-user use. In exchange, because the draft head takes KV cache from the same memory pool, the available context length is cut roughly in half, and it actually becomes slower when handling many concurrent requests<\/strong>. The choice comes down to using MTP enabled for single-user interactive use, and disabled for batching multiple requests.<\/p>\nThe execution kernel repository contains a table with the same intent, but the numbers differ (with a 15.5GiB limit, 1 stream with MTP is 111.2 tok\/s, and KV cache without MTP is 59,753 tokens). This is likely due to differences in settings or versions, but both are measurements by the publisher, and independent third-party replication is not yet available.<\/p>\nStrengths and Use Cases<\/h2>\nThe target use case envisioned by the publisher is agents performing long procedures involving coding, terminal operations, and browser navigation. The publisher’s concern is that even small errors from quantization can cause a single tool call to change, cascading to alter all subsequent states\u2014which is why tasks with long procedures are more susceptible to quantization effects, and why they detailed the “deviation from BF16” so thoroughly.<\/p>\nFrom a local user’s perspective, this model is suited for those who want to set up a 27B-class model as a server using a 16GB GPU (such as an RTX 5080 or 4080)<\/strong>. Since it runs as an OpenAI-compatible API, it can be integrated with coding agents or custom tools.<\/p>\nThere are also many points to keep in mind:<\/p>\n\nYou cannot use a 262K context on 16GB.<\/strong> Because the weights consume about 12.3GB, only about 3 to 4GB remains. The publisher also recommends “starting with a context around 32K on a 16GB GPU and adjusting according to the use case”<\/li>\nImages cannot be read.<\/strong> The visual capabilities of the base Qwen3.8-27B have been stripped out in this version<\/li>\nThe quantization method is private.<\/strong> You cannot apply the exact same procedure to other models yourself<\/li>\nThe publisher is a company operating an AI gateway, and many models released on Hugging Face are uncensored versions of existing models. This OrcaSAQ2 27B is released as a direct quantization of Qwen3.8-27B rather than an uncensored version<\/li>\n<\/ul>\nHow It Differs from Similar Models<\/h2>\n\nDifference from general GGUF quantized versions (Q3_K_M, IQ3 families, etc.)<\/strong>: While the file sizes are similar, the execution engines are completely different. While GGUF runs out of the box in llama.cpp, Ollama, and LM Studio, this version does not run in them (see the next section). Its strengths lie in providing numerical measurements of deviation from BF16 and the ability to handle multiple concurrent requests using vLLM<\/li>\nDifference from other EXL3 quantized versions<\/strong>: The EXL3 format itself is the standard format for ExLlamaV3, and other versions around 3 bits could potentially be created. The differences in OrcaSAQ2 lie in searching for layer-wise bit allocation distributions and providing a plugin to run it with vLLM. However, numerical comparisons against other EXL3 versions under identical conditions are not provided<\/li>\nDifference from the base Qwen3.8-27B (BF16)<\/strong>: The required memory drops to under a quarter, but image input capability is lost, and token-level agreement is limited to 93.2%. The choice is between the original model if accuracy is the top priority, or this version if fitting into 16GB is the priority<\/li>\n<\/ul>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 27.8B parameters (taken from the base model Qwen\/Qwen3.8-27B)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\n16GB (RTX 5060 Ti 16GB \/ 4060 Ti 16GB, etc.)<\/td>\nEXL3<\/td>\n11.4GB<\/td>\n13.7GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nNot usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\nThe publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\nLicense \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The base Qwen3.8-27B model itself is covered in Qwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a>.<\/p>\nSpecifications<\/h2>\n\nBase Model: Qwen3.8-27B (Hybrid architecture of the Qwen3.5 family. Out of 64 layers, 48 are Gated DeltaNet and 16 are standard attention)<\/li>\nQuantization: EXL3 format, averaging 3.21 bits for the decoder part. Output layer (lm_head<\/code>) is 6-bit, embeddings are int8, and MTP heads are 4-bit<\/li>\nDistribution Files: Split into 4 safetensors files, total 12.27GB<\/li>\nContext Length: 262,144 tokens (design upper limit. The length actually usable on a 16GB GPU is described later)<\/li>\nSupported Features: Thinking mode, tool calling, speculative decoding via MTP (Multi-Token Prediction)<\/li>\nInput: Text only. While the base Qwen3.8-27B can read images and videos, this version does not include a visual encoder<\/li>\nLicense: Apache-2.0 (inherited from the base Qwen3.8-27B)<\/li>\n<\/ul>\nPerformance<\/h2>\nDeviation from the Original Model (BF16)<\/h3>\nThe publisher provides values comparing the distributed weights themselves using the same evaluation pipeline as the BF16 version, measured on 16,376 tokens of WikiText-2.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nBuild<\/th>\nSize<\/th>\nDecoder Bit-width<\/th>\nMean KLD \u2193<\/th>\nTop-1 Match Rate \u2191<\/th>\nPPL \u2193<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen3.8-27B BF16<\/td>\n54GB<\/td>\n16<\/td>\n\u2014<\/td>\n100%<\/td>\n5.6468<\/td>\n<\/tr>\n\nOrcaSAQ2 27B<\/td>\n12.3GB<\/td>\n3.21<\/td>\n0.031<\/td>\n93.2%<\/td>\n5.6482<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe meanings of the three metrics are as follows:<\/p>\n\nPPL (Perplexity)<\/strong>: Measures how well the model can predict the continuation of text without confusion. Lower is better. The degradation from BF16 is +0.02%, showing almost no difference<\/li>\nKLD (Kullback-Leibler Divergence)<\/strong>: Measures how much the probability distribution of the next token deviates from BF16. Closer to 0 means more faithful to the original model<\/li>\nTop-1 Match Rate<\/strong>: The proportion of times the “most likely next token chosen” matched the BF16 version<\/li>\n<\/ul>\nIt is important to read these three metrics side by side. Even though the PPL difference is only 0.02%, the Top-1 match rate is 93.2%, meaning roughly once every 15 tokens, a different token is chosen as the most likely candidate compared to BF16<\/strong>. Because PPL is an averaged value across the entire text, individual choices tend to cancel each other out and do not readily show up in the aggregate score. The publisher itself notes in the limitations section that “a 93.2% match rate means some token decisions differ from BF16” and “a +0.02% PPL does not guarantee identical performance on downstream tasks.” Rather than summarizing it as “virtually lossless,” it is more accurate to understand it as “average predictions barely change, but individual decisions change at a certain rate<\/strong>.” Note that comparisons measuring other quantized versions (like GGUF or other EXL3 variants) under the same conditions are not provided, so it is impossible to judge from this data alone whether a KLD of 0.031 is exceptional for the 3-bit range.<\/p>\nAgent Benchmarks (Publisher’s Self-Report)<\/h3>\nThe model card presents scores of 70.0 on SWE-bench Verified and 58.4 on Terminal-Bench 2.1 in a table alongside published values for Claude Sonnet 4.6, Gemini 3, GPT-5.4, and others. However, this article does not treat these as rankings against other models<\/strong> for the following reasons:<\/p>\n\nThe publisher itself clarifies that the table is “not a controlled comparison, but published reference values.” Agent benchmark scores vary significantly depending on the execution environment (harness), thinking budget, and time limits used<\/li>\nThe specific execution environment and settings used to measure the OrcaSAQ2 scores are not documented<\/li>\nThe most crucial piece of information\u2014”the score of the BF16 version measured under the exact same conditions<\/strong>“\u2014is missing. Therefore, it is impossible to tell from this table how much performance was lost due to quantization<\/li>\n<\/ul>\nThe only meaningful evaluation for the quantized version is the “deviation from BF16” discussed in the previous section. It is best not to read more into the agent benchmark scores than simply that the publisher reported them.<\/p>\nSpeed on a 16GB GPU<\/h3>\nValues measured with vLLM while restricting GPU memory to 15.7GiB.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nSetting<\/th>\n1 Stream<\/th>\n8 Streams<\/th>\n16 Streams<\/th>\nKV Cache Capacity<\/th>\n<\/tr>\n<\/thead>\n\n\nWithout MTP<\/td>\n65.3 tok\/s<\/td>\n332 tok\/s<\/td>\n333 tok\/s<\/td>\n29,354 tokens<\/td>\n<\/tr>\n\nWith MTP<\/td>\n90.1 tok\/s<\/td>\n220 tok\/s<\/td>\n219 tok\/s<\/td>\n14,563 tokens<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nUsing MTP (speculative decoding) makes generation about 38% faster for single-user use. In exchange, because the draft head takes KV cache from the same memory pool, the available context length is cut roughly in half, and it actually becomes slower when handling many concurrent requests<\/strong>. The choice comes down to using MTP enabled for single-user interactive use, and disabled for batching multiple requests.<\/p>\nThe execution kernel repository contains a table with the same intent, but the numbers differ (with a 15.5GiB limit, 1 stream with MTP is 111.2 tok\/s, and KV cache without MTP is 59,753 tokens). This is likely due to differences in settings or versions, but both are measurements by the publisher, and independent third-party replication is not yet available.<\/p>\nStrengths and Use Cases<\/h2>\nThe target use case envisioned by the publisher is agents performing long procedures involving coding, terminal operations, and browser navigation. The publisher’s concern is that even small errors from quantization can cause a single tool call to change, cascading to alter all subsequent states\u2014which is why tasks with long procedures are more susceptible to quantization effects, and why they detailed the “deviation from BF16” so thoroughly.<\/p>\nFrom a local user’s perspective, this model is suited for those who want to set up a 27B-class model as a server using a 16GB GPU (such as an RTX 5080 or 4080)<\/strong>. Since it runs as an OpenAI-compatible API, it can be integrated with coding agents or custom tools.<\/p>\nThere are also many points to keep in mind:<\/p>\n\nYou cannot use a 262K context on 16GB.<\/strong> Because the weights consume about 12.3GB, only about 3 to 4GB remains. The publisher also recommends “starting with a context around 32K on a 16GB GPU and adjusting according to the use case”<\/li>\nImages cannot be read.<\/strong> The visual capabilities of the base Qwen3.8-27B have been stripped out in this version<\/li>\nThe quantization method is private.<\/strong> You cannot apply the exact same procedure to other models yourself<\/li>\nThe publisher is a company operating an AI gateway, and many models released on Hugging Face are uncensored versions of existing models. This OrcaSAQ2 27B is released as a direct quantization of Qwen3.8-27B rather than an uncensored version<\/li>\n<\/ul>\nHow It Differs from Similar Models<\/h2>\n\nDifference from general GGUF quantized versions (Q3_K_M, IQ3 families, etc.)<\/strong>: While the file sizes are similar, the execution engines are completely different. While GGUF runs out of the box in llama.cpp, Ollama, and LM Studio, this version does not run in them (see the next section). Its strengths lie in providing numerical measurements of deviation from BF16 and the ability to handle multiple concurrent requests using vLLM<\/li>\nDifference from other EXL3 quantized versions<\/strong>: The EXL3 format itself is the standard format for ExLlamaV3, and other versions around 3 bits could potentially be created. The differences in OrcaSAQ2 lie in searching for layer-wise bit allocation distributions and providing a plugin to run it with vLLM. However, numerical comparisons against other EXL3 versions under identical conditions are not provided<\/li>\nDifference from the base Qwen3.8-27B (BF16)<\/strong>: The required memory drops to under a quarter, but image input capability is lost, and token-level agreement is limited to 93.2%. The choice is between the original model if accuracy is the top priority, or this version if fitting into 16GB is the priority<\/li>\n<\/ul>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 27.8B parameters (taken from the base model Qwen\/Qwen3.8-27B)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\n16GB (RTX 5060 Ti 16GB \/ 4060 Ti 16GB, etc.)<\/td>\nEXL3<\/td>\n11.4GB<\/td>\n13.7GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nNot usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\nThe publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\nLicense \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
lm_head<\/code>) is 6-bit, embeddings are int8, and MTP heads are 4-bit<\/li>\nDistribution Files: Split into 4 safetensors files, total 12.27GB<\/li>\nContext Length: 262,144 tokens (design upper limit. The length actually usable on a 16GB GPU is described later)<\/li>\nSupported Features: Thinking mode, tool calling, speculative decoding via MTP (Multi-Token Prediction)<\/li>\nInput: Text only. While the base Qwen3.8-27B can read images and videos, this version does not include a visual encoder<\/li>\nLicense: Apache-2.0 (inherited from the base Qwen3.8-27B)<\/li>\n<\/ul>\nPerformance<\/h2>\nDeviation from the Original Model (BF16)<\/h3>\nThe publisher provides values comparing the distributed weights themselves using the same evaluation pipeline as the BF16 version, measured on 16,376 tokens of WikiText-2.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nBuild<\/th>\nSize<\/th>\nDecoder Bit-width<\/th>\nMean KLD \u2193<\/th>\nTop-1 Match Rate \u2191<\/th>\nPPL \u2193<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen3.8-27B BF16<\/td>\n54GB<\/td>\n16<\/td>\n\u2014<\/td>\n100%<\/td>\n5.6468<\/td>\n<\/tr>\n\nOrcaSAQ2 27B<\/td>\n12.3GB<\/td>\n3.21<\/td>\n0.031<\/td>\n93.2%<\/td>\n5.6482<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe meanings of the three metrics are as follows:<\/p>\n\nPPL (Perplexity)<\/strong>: Measures how well the model can predict the continuation of text without confusion. Lower is better. The degradation from BF16 is +0.02%, showing almost no difference<\/li>\nKLD (Kullback-Leibler Divergence)<\/strong>: Measures how much the probability distribution of the next token deviates from BF16. Closer to 0 means more faithful to the original model<\/li>\nTop-1 Match Rate<\/strong>: The proportion of times the “most likely next token chosen” matched the BF16 version<\/li>\n<\/ul>\nIt is important to read these three metrics side by side. Even though the PPL difference is only 0.02%, the Top-1 match rate is 93.2%, meaning roughly once every 15 tokens, a different token is chosen as the most likely candidate compared to BF16<\/strong>. Because PPL is an averaged value across the entire text, individual choices tend to cancel each other out and do not readily show up in the aggregate score. The publisher itself notes in the limitations section that “a 93.2% match rate means some token decisions differ from BF16” and “a +0.02% PPL does not guarantee identical performance on downstream tasks.” Rather than summarizing it as “virtually lossless,” it is more accurate to understand it as “average predictions barely change, but individual decisions change at a certain rate<\/strong>.” Note that comparisons measuring other quantized versions (like GGUF or other EXL3 variants) under the same conditions are not provided, so it is impossible to judge from this data alone whether a KLD of 0.031 is exceptional for the 3-bit range.<\/p>\nAgent Benchmarks (Publisher’s Self-Report)<\/h3>\nThe model card presents scores of 70.0 on SWE-bench Verified and 58.4 on Terminal-Bench 2.1 in a table alongside published values for Claude Sonnet 4.6, Gemini 3, GPT-5.4, and others. However, this article does not treat these as rankings against other models<\/strong> for the following reasons:<\/p>\n\nThe publisher itself clarifies that the table is “not a controlled comparison, but published reference values.” Agent benchmark scores vary significantly depending on the execution environment (harness), thinking budget, and time limits used<\/li>\nThe specific execution environment and settings used to measure the OrcaSAQ2 scores are not documented<\/li>\nThe most crucial piece of information\u2014”the score of the BF16 version measured under the exact same conditions<\/strong>“\u2014is missing. Therefore, it is impossible to tell from this table how much performance was lost due to quantization<\/li>\n<\/ul>\nThe only meaningful evaluation for the quantized version is the “deviation from BF16” discussed in the previous section. It is best not to read more into the agent benchmark scores than simply that the publisher reported them.<\/p>\nSpeed on a 16GB GPU<\/h3>\nValues measured with vLLM while restricting GPU memory to 15.7GiB.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nSetting<\/th>\n1 Stream<\/th>\n8 Streams<\/th>\n16 Streams<\/th>\nKV Cache Capacity<\/th>\n<\/tr>\n<\/thead>\n\n\nWithout MTP<\/td>\n65.3 tok\/s<\/td>\n332 tok\/s<\/td>\n333 tok\/s<\/td>\n29,354 tokens<\/td>\n<\/tr>\n\nWith MTP<\/td>\n90.1 tok\/s<\/td>\n220 tok\/s<\/td>\n219 tok\/s<\/td>\n14,563 tokens<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nUsing MTP (speculative decoding) makes generation about 38% faster for single-user use. In exchange, because the draft head takes KV cache from the same memory pool, the available context length is cut roughly in half, and it actually becomes slower when handling many concurrent requests<\/strong>. The choice comes down to using MTP enabled for single-user interactive use, and disabled for batching multiple requests.<\/p>\nThe execution kernel repository contains a table with the same intent, but the numbers differ (with a 15.5GiB limit, 1 stream with MTP is 111.2 tok\/s, and KV cache without MTP is 59,753 tokens). This is likely due to differences in settings or versions, but both are measurements by the publisher, and independent third-party replication is not yet available.<\/p>\nStrengths and Use Cases<\/h2>\nThe target use case envisioned by the publisher is agents performing long procedures involving coding, terminal operations, and browser navigation. The publisher’s concern is that even small errors from quantization can cause a single tool call to change, cascading to alter all subsequent states\u2014which is why tasks with long procedures are more susceptible to quantization effects, and why they detailed the “deviation from BF16” so thoroughly.<\/p>\nFrom a local user’s perspective, this model is suited for those who want to set up a 27B-class model as a server using a 16GB GPU (such as an RTX 5080 or 4080)<\/strong>. Since it runs as an OpenAI-compatible API, it can be integrated with coding agents or custom tools.<\/p>\nThere are also many points to keep in mind:<\/p>\n\nYou cannot use a 262K context on 16GB.<\/strong> Because the weights consume about 12.3GB, only about 3 to 4GB remains. The publisher also recommends “starting with a context around 32K on a 16GB GPU and adjusting according to the use case”<\/li>\nImages cannot be read.<\/strong> The visual capabilities of the base Qwen3.8-27B have been stripped out in this version<\/li>\nThe quantization method is private.<\/strong> You cannot apply the exact same procedure to other models yourself<\/li>\nThe publisher is a company operating an AI gateway, and many models released on Hugging Face are uncensored versions of existing models. This OrcaSAQ2 27B is released as a direct quantization of Qwen3.8-27B rather than an uncensored version<\/li>\n<\/ul>\nHow It Differs from Similar Models<\/h2>\n\nDifference from general GGUF quantized versions (Q3_K_M, IQ3 families, etc.)<\/strong>: While the file sizes are similar, the execution engines are completely different. While GGUF runs out of the box in llama.cpp, Ollama, and LM Studio, this version does not run in them (see the next section). Its strengths lie in providing numerical measurements of deviation from BF16 and the ability to handle multiple concurrent requests using vLLM<\/li>\nDifference from other EXL3 quantized versions<\/strong>: The EXL3 format itself is the standard format for ExLlamaV3, and other versions around 3 bits could potentially be created. The differences in OrcaSAQ2 lie in searching for layer-wise bit allocation distributions and providing a plugin to run it with vLLM. However, numerical comparisons against other EXL3 versions under identical conditions are not provided<\/li>\nDifference from the base Qwen3.8-27B (BF16)<\/strong>: The required memory drops to under a quarter, but image input capability is lost, and token-level agreement is limited to 93.2%. The choice is between the original model if accuracy is the top priority, or this version if fitting into 16GB is the priority<\/li>\n<\/ul>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 27.8B parameters (taken from the base model Qwen\/Qwen3.8-27B)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\n16GB (RTX 5060 Ti 16GB \/ 4060 Ti 16GB, etc.)<\/td>\nEXL3<\/td>\n11.4GB<\/td>\n13.7GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nNot usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\nThe publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\nLicense \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The publisher provides values comparing the distributed weights themselves using the same evaluation pipeline as the BF16 version, measured on 16,376 tokens of WikiText-2.<\/p>\n
\u2192 Scroll horizontally to see all columns<\/p>\n
The meanings of the three metrics are as follows:<\/p>\n
It is important to read these three metrics side by side. Even though the PPL difference is only 0.02%, the Top-1 match rate is 93.2%, meaning roughly once every 15 tokens, a different token is chosen as the most likely candidate compared to BF16<\/strong>. Because PPL is an averaged value across the entire text, individual choices tend to cancel each other out and do not readily show up in the aggregate score. The publisher itself notes in the limitations section that “a 93.2% match rate means some token decisions differ from BF16” and “a +0.02% PPL does not guarantee identical performance on downstream tasks.” Rather than summarizing it as “virtually lossless,” it is more accurate to understand it as “average predictions barely change, but individual decisions change at a certain rate<\/strong>.” Note that comparisons measuring other quantized versions (like GGUF or other EXL3 variants) under the same conditions are not provided, so it is impossible to judge from this data alone whether a KLD of 0.031 is exceptional for the 3-bit range.<\/p>\nAgent Benchmarks (Publisher’s Self-Report)<\/h3>\nThe model card presents scores of 70.0 on SWE-bench Verified and 58.4 on Terminal-Bench 2.1 in a table alongside published values for Claude Sonnet 4.6, Gemini 3, GPT-5.4, and others. However, this article does not treat these as rankings against other models<\/strong> for the following reasons:<\/p>\n\nThe publisher itself clarifies that the table is “not a controlled comparison, but published reference values.” Agent benchmark scores vary significantly depending on the execution environment (harness), thinking budget, and time limits used<\/li>\nThe specific execution environment and settings used to measure the OrcaSAQ2 scores are not documented<\/li>\nThe most crucial piece of information\u2014”the score of the BF16 version measured under the exact same conditions<\/strong>“\u2014is missing. Therefore, it is impossible to tell from this table how much performance was lost due to quantization<\/li>\n<\/ul>\nThe only meaningful evaluation for the quantized version is the “deviation from BF16” discussed in the previous section. It is best not to read more into the agent benchmark scores than simply that the publisher reported them.<\/p>\nSpeed on a 16GB GPU<\/h3>\nValues measured with vLLM while restricting GPU memory to 15.7GiB.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nSetting<\/th>\n1 Stream<\/th>\n8 Streams<\/th>\n16 Streams<\/th>\nKV Cache Capacity<\/th>\n<\/tr>\n<\/thead>\n\n\nWithout MTP<\/td>\n65.3 tok\/s<\/td>\n332 tok\/s<\/td>\n333 tok\/s<\/td>\n29,354 tokens<\/td>\n<\/tr>\n\nWith MTP<\/td>\n90.1 tok\/s<\/td>\n220 tok\/s<\/td>\n219 tok\/s<\/td>\n14,563 tokens<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nUsing MTP (speculative decoding) makes generation about 38% faster for single-user use. In exchange, because the draft head takes KV cache from the same memory pool, the available context length is cut roughly in half, and it actually becomes slower when handling many concurrent requests<\/strong>. The choice comes down to using MTP enabled for single-user interactive use, and disabled for batching multiple requests.<\/p>\nThe execution kernel repository contains a table with the same intent, but the numbers differ (with a 15.5GiB limit, 1 stream with MTP is 111.2 tok\/s, and KV cache without MTP is 59,753 tokens). This is likely due to differences in settings or versions, but both are measurements by the publisher, and independent third-party replication is not yet available.<\/p>\nStrengths and Use Cases<\/h2>\nThe target use case envisioned by the publisher is agents performing long procedures involving coding, terminal operations, and browser navigation. The publisher’s concern is that even small errors from quantization can cause a single tool call to change, cascading to alter all subsequent states\u2014which is why tasks with long procedures are more susceptible to quantization effects, and why they detailed the “deviation from BF16” so thoroughly.<\/p>\nFrom a local user’s perspective, this model is suited for those who want to set up a 27B-class model as a server using a 16GB GPU (such as an RTX 5080 or 4080)<\/strong>. Since it runs as an OpenAI-compatible API, it can be integrated with coding agents or custom tools.<\/p>\nThere are also many points to keep in mind:<\/p>\n\nYou cannot use a 262K context on 16GB.<\/strong> Because the weights consume about 12.3GB, only about 3 to 4GB remains. The publisher also recommends “starting with a context around 32K on a 16GB GPU and adjusting according to the use case”<\/li>\nImages cannot be read.<\/strong> The visual capabilities of the base Qwen3.8-27B have been stripped out in this version<\/li>\nThe quantization method is private.<\/strong> You cannot apply the exact same procedure to other models yourself<\/li>\nThe publisher is a company operating an AI gateway, and many models released on Hugging Face are uncensored versions of existing models. This OrcaSAQ2 27B is released as a direct quantization of Qwen3.8-27B rather than an uncensored version<\/li>\n<\/ul>\nHow It Differs from Similar Models<\/h2>\n\nDifference from general GGUF quantized versions (Q3_K_M, IQ3 families, etc.)<\/strong>: While the file sizes are similar, the execution engines are completely different. While GGUF runs out of the box in llama.cpp, Ollama, and LM Studio, this version does not run in them (see the next section). Its strengths lie in providing numerical measurements of deviation from BF16 and the ability to handle multiple concurrent requests using vLLM<\/li>\nDifference from other EXL3 quantized versions<\/strong>: The EXL3 format itself is the standard format for ExLlamaV3, and other versions around 3 bits could potentially be created. The differences in OrcaSAQ2 lie in searching for layer-wise bit allocation distributions and providing a plugin to run it with vLLM. However, numerical comparisons against other EXL3 versions under identical conditions are not provided<\/li>\nDifference from the base Qwen3.8-27B (BF16)<\/strong>: The required memory drops to under a quarter, but image input capability is lost, and token-level agreement is limited to 93.2%. The choice is between the original model if accuracy is the top priority, or this version if fitting into 16GB is the priority<\/li>\n<\/ul>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 27.8B parameters (taken from the base model Qwen\/Qwen3.8-27B)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\n16GB (RTX 5060 Ti 16GB \/ 4060 Ti 16GB, etc.)<\/td>\nEXL3<\/td>\n11.4GB<\/td>\n13.7GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nNot usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\nThe publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\nLicense \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The model card presents scores of 70.0 on SWE-bench Verified and 58.4 on Terminal-Bench 2.1 in a table alongside published values for Claude Sonnet 4.6, Gemini 3, GPT-5.4, and others. However, this article does not treat these as rankings against other models<\/strong> for the following reasons:<\/p>\n\nThe publisher itself clarifies that the table is “not a controlled comparison, but published reference values.” Agent benchmark scores vary significantly depending on the execution environment (harness), thinking budget, and time limits used<\/li>\nThe specific execution environment and settings used to measure the OrcaSAQ2 scores are not documented<\/li>\nThe most crucial piece of information\u2014”the score of the BF16 version measured under the exact same conditions<\/strong>“\u2014is missing. Therefore, it is impossible to tell from this table how much performance was lost due to quantization<\/li>\n<\/ul>\nThe only meaningful evaluation for the quantized version is the “deviation from BF16” discussed in the previous section. It is best not to read more into the agent benchmark scores than simply that the publisher reported them.<\/p>\nSpeed on a 16GB GPU<\/h3>\nValues measured with vLLM while restricting GPU memory to 15.7GiB.<\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nSetting<\/th>\n1 Stream<\/th>\n8 Streams<\/th>\n16 Streams<\/th>\nKV Cache Capacity<\/th>\n<\/tr>\n<\/thead>\n\n\nWithout MTP<\/td>\n65.3 tok\/s<\/td>\n332 tok\/s<\/td>\n333 tok\/s<\/td>\n29,354 tokens<\/td>\n<\/tr>\n\nWith MTP<\/td>\n90.1 tok\/s<\/td>\n220 tok\/s<\/td>\n219 tok\/s<\/td>\n14,563 tokens<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nUsing MTP (speculative decoding) makes generation about 38% faster for single-user use. In exchange, because the draft head takes KV cache from the same memory pool, the available context length is cut roughly in half, and it actually becomes slower when handling many concurrent requests<\/strong>. The choice comes down to using MTP enabled for single-user interactive use, and disabled for batching multiple requests.<\/p>\nThe execution kernel repository contains a table with the same intent, but the numbers differ (with a 15.5GiB limit, 1 stream with MTP is 111.2 tok\/s, and KV cache without MTP is 59,753 tokens). This is likely due to differences in settings or versions, but both are measurements by the publisher, and independent third-party replication is not yet available.<\/p>\nStrengths and Use Cases<\/h2>\nThe target use case envisioned by the publisher is agents performing long procedures involving coding, terminal operations, and browser navigation. The publisher’s concern is that even small errors from quantization can cause a single tool call to change, cascading to alter all subsequent states\u2014which is why tasks with long procedures are more susceptible to quantization effects, and why they detailed the “deviation from BF16” so thoroughly.<\/p>\nFrom a local user’s perspective, this model is suited for those who want to set up a 27B-class model as a server using a 16GB GPU (such as an RTX 5080 or 4080)<\/strong>. Since it runs as an OpenAI-compatible API, it can be integrated with coding agents or custom tools.<\/p>\nThere are also many points to keep in mind:<\/p>\n\nYou cannot use a 262K context on 16GB.<\/strong> Because the weights consume about 12.3GB, only about 3 to 4GB remains. The publisher also recommends “starting with a context around 32K on a 16GB GPU and adjusting according to the use case”<\/li>\nImages cannot be read.<\/strong> The visual capabilities of the base Qwen3.8-27B have been stripped out in this version<\/li>\nThe quantization method is private.<\/strong> You cannot apply the exact same procedure to other models yourself<\/li>\nThe publisher is a company operating an AI gateway, and many models released on Hugging Face are uncensored versions of existing models. This OrcaSAQ2 27B is released as a direct quantization of Qwen3.8-27B rather than an uncensored version<\/li>\n<\/ul>\nHow It Differs from Similar Models<\/h2>\n\nDifference from general GGUF quantized versions (Q3_K_M, IQ3 families, etc.)<\/strong>: While the file sizes are similar, the execution engines are completely different. While GGUF runs out of the box in llama.cpp, Ollama, and LM Studio, this version does not run in them (see the next section). Its strengths lie in providing numerical measurements of deviation from BF16 and the ability to handle multiple concurrent requests using vLLM<\/li>\nDifference from other EXL3 quantized versions<\/strong>: The EXL3 format itself is the standard format for ExLlamaV3, and other versions around 3 bits could potentially be created. The differences in OrcaSAQ2 lie in searching for layer-wise bit allocation distributions and providing a plugin to run it with vLLM. However, numerical comparisons against other EXL3 versions under identical conditions are not provided<\/li>\nDifference from the base Qwen3.8-27B (BF16)<\/strong>: The required memory drops to under a quarter, but image input capability is lost, and token-level agreement is limited to 93.2%. The choice is between the original model if accuracy is the top priority, or this version if fitting into 16GB is the priority<\/li>\n<\/ul>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 27.8B parameters (taken from the base model Qwen\/Qwen3.8-27B)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\n16GB (RTX 5060 Ti 16GB \/ 4060 Ti 16GB, etc.)<\/td>\nEXL3<\/td>\n11.4GB<\/td>\n13.7GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nNot usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\nThe publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\nLicense \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The only meaningful evaluation for the quantized version is the “deviation from BF16” discussed in the previous section. It is best not to read more into the agent benchmark scores than simply that the publisher reported them.<\/p>\n
Values measured with vLLM while restricting GPU memory to 15.7GiB.<\/p>\n
Using MTP (speculative decoding) makes generation about 38% faster for single-user use. In exchange, because the draft head takes KV cache from the same memory pool, the available context length is cut roughly in half, and it actually becomes slower when handling many concurrent requests<\/strong>. The choice comes down to using MTP enabled for single-user interactive use, and disabled for batching multiple requests.<\/p>\nThe execution kernel repository contains a table with the same intent, but the numbers differ (with a 15.5GiB limit, 1 stream with MTP is 111.2 tok\/s, and KV cache without MTP is 59,753 tokens). This is likely due to differences in settings or versions, but both are measurements by the publisher, and independent third-party replication is not yet available.<\/p>\nStrengths and Use Cases<\/h2>\nThe target use case envisioned by the publisher is agents performing long procedures involving coding, terminal operations, and browser navigation. The publisher’s concern is that even small errors from quantization can cause a single tool call to change, cascading to alter all subsequent states\u2014which is why tasks with long procedures are more susceptible to quantization effects, and why they detailed the “deviation from BF16” so thoroughly.<\/p>\nFrom a local user’s perspective, this model is suited for those who want to set up a 27B-class model as a server using a 16GB GPU (such as an RTX 5080 or 4080)<\/strong>. Since it runs as an OpenAI-compatible API, it can be integrated with coding agents or custom tools.<\/p>\nThere are also many points to keep in mind:<\/p>\n\nYou cannot use a 262K context on 16GB.<\/strong> Because the weights consume about 12.3GB, only about 3 to 4GB remains. The publisher also recommends “starting with a context around 32K on a 16GB GPU and adjusting according to the use case”<\/li>\nImages cannot be read.<\/strong> The visual capabilities of the base Qwen3.8-27B have been stripped out in this version<\/li>\nThe quantization method is private.<\/strong> You cannot apply the exact same procedure to other models yourself<\/li>\nThe publisher is a company operating an AI gateway, and many models released on Hugging Face are uncensored versions of existing models. This OrcaSAQ2 27B is released as a direct quantization of Qwen3.8-27B rather than an uncensored version<\/li>\n<\/ul>\nHow It Differs from Similar Models<\/h2>\n\nDifference from general GGUF quantized versions (Q3_K_M, IQ3 families, etc.)<\/strong>: While the file sizes are similar, the execution engines are completely different. While GGUF runs out of the box in llama.cpp, Ollama, and LM Studio, this version does not run in them (see the next section). Its strengths lie in providing numerical measurements of deviation from BF16 and the ability to handle multiple concurrent requests using vLLM<\/li>\nDifference from other EXL3 quantized versions<\/strong>: The EXL3 format itself is the standard format for ExLlamaV3, and other versions around 3 bits could potentially be created. The differences in OrcaSAQ2 lie in searching for layer-wise bit allocation distributions and providing a plugin to run it with vLLM. However, numerical comparisons against other EXL3 versions under identical conditions are not provided<\/li>\nDifference from the base Qwen3.8-27B (BF16)<\/strong>: The required memory drops to under a quarter, but image input capability is lost, and token-level agreement is limited to 93.2%. The choice is between the original model if accuracy is the top priority, or this version if fitting into 16GB is the priority<\/li>\n<\/ul>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 27.8B parameters (taken from the base model Qwen\/Qwen3.8-27B)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\n16GB (RTX 5060 Ti 16GB \/ 4060 Ti 16GB, etc.)<\/td>\nEXL3<\/td>\n11.4GB<\/td>\n13.7GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nNot usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\nThe publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\nLicense \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The execution kernel repository contains a table with the same intent, but the numbers differ (with a 15.5GiB limit, 1 stream with MTP is 111.2 tok\/s, and KV cache without MTP is 59,753 tokens). This is likely due to differences in settings or versions, but both are measurements by the publisher, and independent third-party replication is not yet available.<\/p>\n
The target use case envisioned by the publisher is agents performing long procedures involving coding, terminal operations, and browser navigation. The publisher’s concern is that even small errors from quantization can cause a single tool call to change, cascading to alter all subsequent states\u2014which is why tasks with long procedures are more susceptible to quantization effects, and why they detailed the “deviation from BF16” so thoroughly.<\/p>\n
From a local user’s perspective, this model is suited for those who want to set up a 27B-class model as a server using a 16GB GPU (such as an RTX 5080 or 4080)<\/strong>. Since it runs as an OpenAI-compatible API, it can be integrated with coding agents or custom tools.<\/p>\nThere are also many points to keep in mind:<\/p>\n\nYou cannot use a 262K context on 16GB.<\/strong> Because the weights consume about 12.3GB, only about 3 to 4GB remains. The publisher also recommends “starting with a context around 32K on a 16GB GPU and adjusting according to the use case”<\/li>\nImages cannot be read.<\/strong> The visual capabilities of the base Qwen3.8-27B have been stripped out in this version<\/li>\nThe quantization method is private.<\/strong> You cannot apply the exact same procedure to other models yourself<\/li>\nThe publisher is a company operating an AI gateway, and many models released on Hugging Face are uncensored versions of existing models. This OrcaSAQ2 27B is released as a direct quantization of Qwen3.8-27B rather than an uncensored version<\/li>\n<\/ul>\nHow It Differs from Similar Models<\/h2>\n\nDifference from general GGUF quantized versions (Q3_K_M, IQ3 families, etc.)<\/strong>: While the file sizes are similar, the execution engines are completely different. While GGUF runs out of the box in llama.cpp, Ollama, and LM Studio, this version does not run in them (see the next section). Its strengths lie in providing numerical measurements of deviation from BF16 and the ability to handle multiple concurrent requests using vLLM<\/li>\nDifference from other EXL3 quantized versions<\/strong>: The EXL3 format itself is the standard format for ExLlamaV3, and other versions around 3 bits could potentially be created. The differences in OrcaSAQ2 lie in searching for layer-wise bit allocation distributions and providing a plugin to run it with vLLM. However, numerical comparisons against other EXL3 versions under identical conditions are not provided<\/li>\nDifference from the base Qwen3.8-27B (BF16)<\/strong>: The required memory drops to under a quarter, but image input capability is lost, and token-level agreement is limited to 93.2%. The choice is between the original model if accuracy is the top priority, or this version if fitting into 16GB is the priority<\/li>\n<\/ul>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 27.8B parameters (taken from the base model Qwen\/Qwen3.8-27B)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\n16GB (RTX 5060 Ti 16GB \/ 4060 Ti 16GB, etc.)<\/td>\nEXL3<\/td>\n11.4GB<\/td>\n13.7GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nNot usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\nThe publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\nLicense \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
There are also many points to keep in mind:<\/p>\n
Estimated requirements (calculated by Local Model Watch)<\/strong> \u2014 27.8B parameters (taken from the base model Qwen\/Qwen3.8-27B)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\n16GB (RTX 5060 Ti 16GB \/ 4060 Ti 16GB, etc.)<\/td>\nEXL3<\/td>\n11.4GB<\/td>\n13.7GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nNot usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\nThe publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\nLicense \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nNot usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\nThe publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\nLicense \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Not usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\nThe publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\nLicense \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\n
License \u2014 apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Values we measured ourselves on our server (no GPU) by actually reading and running this model’s files \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n546<\/strong><\/td>\n0.99\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\nCounted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
It needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\n
Counted with the tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
tokenizer.json<\/code> of orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Models with <\/em>15\u201340B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nXingChen-AGI\/Xing4.0-29B-A4B<\/td>\n31.2B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nXing4.0-29B-A4B 29B MoE Model Strong in Coding Agents: 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n\nEdge0\/Edge0-35B-A3B-preview<\/td>\n36.0B<\/td>\n24GB<\/td>\napache-2.0<\/td>\nEdge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nbartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n26.5B<\/td>\n12GB<\/td>\napache-2.0<\/td>\nGryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: 12GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n\nnex-agi\/Nex-N2.5-mini<\/td>\n35.1B<\/td>\n16GB<\/td>\napache-2.0<\/td>\nNex-N2.5-mini Agent Model for Long-Horizon Tasks: 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\n\nDistribution: Hugging Face orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\nDoes not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: “Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer”<\/li>\nWhen running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher’s plugin (Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append --reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
--reasoning-parser qwen3<\/code> and --tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
--tool-call-parser qwen3_xml<\/code><\/li>\nWhen running with ExLlamaV3<\/strong>: The kernel repository notes that “for single-user use on consumer GPUs, it is lighter than vLLM and better suited.” However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\nRecommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\nThe execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\nLicense: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<\/p>\nRelated Articles<\/h2>\n\nHemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\nSignal-3.8-27B-GGUF Token-Efficient Optimized GGUF Model: 16GB+ VRAM<\/a><\/li>\nQwen3.8-27B Multimodal Vision-Language Model: 8GB+ VRAM, GGUF Builds<\/a><\/li>\nQwen3.8-27B TWIN-TURBO Uncensored GGUF Released<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 Other models that run on a 16GB GPU<\/a><\/li>\nExplore the same model family<\/strong> \u2192 Qwen3.8 family overview (8 articles, 11 converted builds)<\/a><\/li>\nWhat EXL3 mean and where to get this model<\/strong> \u2192 EXL2 \/ EXL3 format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Alibaba (Qwen): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other text generation models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nhttps:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\nhttps:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\nhttps:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<\/p>\n\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\nThis article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
This article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":6422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1565,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-unverified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":6803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/6803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/6422"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}