{"id":6423,"date":"2026-09-28T13:50:34","date_gmt":"2026-09-28T04:50:34","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/09\/28\/orcasaq2-27b-qwen3-8-27b-quantized\/"},"modified":"2026-11-11T00:36:02","modified_gmt":"2026-11-10T15:36:02","slug":"orcasaq2-27b-qwen3-8-27b-quantized","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/28\/orcasaq2-27b-qwen3-8-27b-quantized\/","title":{"rendered":"OrcaSAQ-2-27B Text Generation Model: Our Test Answers, 16GB+ VRAM"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B\">orcarouter\/OrcaSAQ-2-27B<\/a><\/td>\n<\/tr>\n<tr>\n<td>Family guide<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/model-qwen-qwen3-8-en\/\">Qwen3.8 guide (8 articles)<\/a><\/td>\n<\/tr>\n<tr>\n<td>Publisher guide<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/publisher-alibaba-en\/\">Alibaba (Qwen): models and licenses<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-09-24<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>apache-2.0<\/td>\n<\/tr>\n<tr>\n<td>Formats<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-exl-en\/\">EXL3<\/a> \/ safetensors<\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<p><!-- lmw:lab-summary --><\/p>\n<p><strong>What we checked ourselves<\/strong><\/p>\n<ul>\n<li>Ran the publisher&#8217;s own code (transformers , , revision <code>15d20d7e9a<\/code>) on a cloud GPU on Modal (NVIDIA L4, peak VRAM 14.0GB) and show its answers verbatim.<\/li>\n<li>It needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/li>\n<\/ul>\n<p>Details and conditions are in \u201cOur Own Measurements\u201d below.<\/p>\n<p><!-- \/lmw:lab-summary --><\/p>\n<h2>Overview<\/h2>\n<p>OrcaRouter, an AI gateway operator, has released a quantized version of Qwen3.8-27B compressed to an average of 3.21 bits, named <strong>OrcaSAQ2 27B<\/strong> (<code>orcarouter\/OrcaSAQ-2-27B<\/code>), on Hugging Face. The original model, which is 54GB in BF16, has been reduced to a total of 12.3GB across its distribution files. The stated goal of the publisher is to &#8220;<strong>run a 27B model on a single 16GB GPU<\/strong>,&#8221; and they present numerical comparisons directly against the BF16 version to show that the difference from the original model is small.<\/p>\n<p>The publisher calls this a &#8220;proprietary sensitivity-aware mixed-precision quantization system (SAQ2)&#8221; and has not disclosed the details of the method (calibration procedures, bit-width allocation per layer, or packing methods). However, looking at the distribution files&#8217; configuration (<code>quantization_config.json<\/code>), the quantization format is <code>exl3<\/code> (ExLlamaV3 format, version 1.5.1). The publisher&#8217;s concurrently released execution kernel repository also describes its contents as &#8220;a combination of EXL3 trellis quantization (QTIP family) with mixed-precision allocation determined by search.&#8221; In other words, it is accurate to read this as <strong>not a new format, but rather a clever allocation of bit-widths per layer within the existing EXL3 format<\/strong>.<\/p>\n<p>The base Qwen3.8-27B model itself is covered in <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/26\/qwen3-8-27b-overview-specs-performance\/\">Qwen3.8-27B Vision-Language Model: Our Test Answers, 8GB+ VRAM<\/a>.<\/p>\n<h2>Specifications<\/h2>\n<ul>\n<li>Base Model: Qwen3.8-27B (Hybrid architecture of the Qwen3.5 family. Out of 64 layers, 48 are Gated DeltaNet and 16 are standard attention)<\/li>\n<li>Quantization: EXL3 format, averaging 3.21 bits for the decoder part. Output layer (<code>lm_head<\/code>) is 6-bit, embeddings are int8, and MTP heads are 4-bit<\/li>\n<li>Distribution Files: Split into 4 safetensors files, total 12.27GB<\/li>\n<li>Context Length: 262,144 tokens (design upper limit. The length actually usable on a 16GB GPU is described later)<\/li>\n<li>Supported Features: Thinking mode, tool calling, speculative decoding via MTP (Multi-Token Prediction)<\/li>\n<li>Input: Text only. While the base Qwen3.8-27B can read images and videos, this version does not include a visual encoder<\/li>\n<li>License: Apache-2.0 (inherited from the base Qwen3.8-27B)<\/li>\n<\/ul>\n<h2>Performance<\/h2>\n<h3>Deviation from the Original Model (BF16)<\/h3>\n<p>The publisher provides values comparing the distributed weights themselves using the same evaluation pipeline as the BF16 version, measured on 16,376 tokens of WikiText-2.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Build<\/th>\n<th>Size<\/th>\n<th>Decoder Bit-width<\/th>\n<th>Mean KLD \u2193<\/th>\n<th>Top-1 Match Rate \u2191<\/th>\n<th>PPL \u2193<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Qwen3.8-27B BF16<\/td>\n<td>54GB<\/td>\n<td>16<\/td>\n<td>\u2014<\/td>\n<td>100%<\/td>\n<td>5.6468<\/td>\n<\/tr>\n<tr>\n<td>OrcaSAQ2 27B<\/td>\n<td>12.3GB<\/td>\n<td>3.21<\/td>\n<td>0.031<\/td>\n<td>93.2%<\/td>\n<td>5.6482<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>The meanings of the three metrics are as follows:<\/p>\n<ul>\n<li><strong>PPL (Perplexity)<\/strong>: Measures how well the model can predict the continuation of text without confusion. Lower is better. The degradation from BF16 is +0.02%, showing almost no difference<\/li>\n<li><strong>KLD (Kullback-Leibler Divergence)<\/strong>: Measures how much the probability distribution of the next token deviates from BF16. Closer to 0 means more faithful to the original model<\/li>\n<li><strong>Top-1 Match Rate<\/strong>: The proportion of times the &#8220;most likely next token chosen&#8221; matched the BF16 version<\/li>\n<\/ul>\n<p>It is important to read these three metrics side by side. Even though the PPL difference is only 0.02%, the Top-1 match rate is 93.2%, meaning <strong>roughly once every 15 tokens, a different token is chosen as the most likely candidate compared to BF16<\/strong>. Because PPL is an averaged value across the entire text, individual choices tend to cancel each other out and do not readily show up in the aggregate score. The publisher itself notes in the limitations section that &#8220;a 93.2% match rate means some token decisions differ from BF16&#8221; and &#8220;a +0.02% PPL does not guarantee identical performance on downstream tasks.&#8221; Rather than summarizing it as &#8220;virtually lossless,&#8221; it is more accurate to understand it as &#8220;<strong>average predictions barely change, but individual decisions change at a certain rate<\/strong>.&#8221; Note that comparisons measuring other quantized versions (like GGUF or other EXL3 variants) under the same conditions are not provided, so it is impossible to judge from this data alone whether a KLD of 0.031 is exceptional for the 3-bit range.<\/p>\n<h3>Agent Benchmarks (Publisher&#8217;s Self-Report)<\/h3>\n<p>The model card presents scores of 70.0 on SWE-bench Verified and 58.4 on Terminal-Bench 2.1 in a table alongside published values for Claude Sonnet 4.6, Gemini 3, GPT-5.4, and others. However, <strong>this article does not treat these as rankings against other models<\/strong> for the following reasons:<\/p>\n<ul>\n<li>The publisher itself clarifies that the table is &#8220;not a controlled comparison, but published reference values.&#8221; Agent benchmark scores vary significantly depending on the execution environment (harness), thinking budget, and time limits used<\/li>\n<li>The specific execution environment and settings used to measure the OrcaSAQ2 scores are not documented<\/li>\n<li>The most crucial piece of information\u2014&#8221;<strong>the score of the BF16 version measured under the exact same conditions<\/strong>&#8220;\u2014is missing. Therefore, it is impossible to tell from this table how much performance was lost due to quantization<\/li>\n<\/ul>\n<p>The only meaningful evaluation for the quantized version is the &#8220;deviation from BF16&#8221; discussed in the previous section. It is best not to read more into the agent benchmark scores than simply that the publisher reported them.<\/p>\n<h3>Speed on a 16GB GPU<\/h3>\n<p>Values measured with vLLM while restricting GPU memory to 15.7GiB.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Setting<\/th>\n<th>1 Stream<\/th>\n<th>8 Streams<\/th>\n<th>16 Streams<\/th>\n<th>KV Cache Capacity<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Without MTP<\/td>\n<td>65.3 tok\/s<\/td>\n<td>332 tok\/s<\/td>\n<td>333 tok\/s<\/td>\n<td>29,354 tokens<\/td>\n<\/tr>\n<tr>\n<td>With MTP<\/td>\n<td>90.1 tok\/s<\/td>\n<td>220 tok\/s<\/td>\n<td>219 tok\/s<\/td>\n<td>14,563 tokens<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>Using MTP (speculative decoding) makes generation about 38% faster for single-user use. In exchange, because the draft head takes KV cache from the same memory pool, <strong>the available context length is cut roughly in half, and it actually becomes slower when handling many concurrent requests<\/strong>. The choice comes down to using MTP enabled for single-user interactive use, and disabled for batching multiple requests.<\/p>\n<p>The execution kernel repository contains a table with the same intent, but the numbers differ (with a 15.5GiB limit, 1 stream with MTP is 111.2 tok\/s, and KV cache without MTP is 59,753 tokens). This is likely due to differences in settings or versions, but both are measurements by the publisher, and independent third-party replication is not yet available.<\/p>\n<h2>Strengths and Use Cases<\/h2>\n<p>The target use case envisioned by the publisher is agents performing long procedures involving coding, terminal operations, and browser navigation. The publisher&#8217;s concern is that even small errors from quantization can cause a single tool call to change, cascading to alter all subsequent states\u2014which is why tasks with long procedures are more susceptible to quantization effects, and why they detailed the &#8220;deviation from BF16&#8221; so thoroughly.<\/p>\n<p>From a local user&#8217;s perspective, this model is suited for those who want to <strong>set up a 27B-class model as a server using a 16GB GPU (such as an RTX 5080 or 4080)<\/strong>. Since it runs as an OpenAI-compatible API, it can be integrated with coding agents or custom tools.<\/p>\n<p>There are also many points to keep in mind:<\/p>\n<ul>\n<li><strong>You cannot use a 262K context on 16GB.<\/strong> Because the weights consume about 12.3GB, only about 3 to 4GB remains. The publisher also recommends &#8220;starting with a context around 32K on a 16GB GPU and adjusting according to the use case&#8221;<\/li>\n<li><strong>Images cannot be read.<\/strong> The visual capabilities of the base Qwen3.8-27B have been stripped out in this version<\/li>\n<li><strong>The quantization method is private.<\/strong> You cannot apply the exact same procedure to other models yourself<\/li>\n<li>The publisher is a company operating an AI gateway, and many models released on Hugging Face are uncensored versions of existing models. This OrcaSAQ2 27B is released as a direct quantization of Qwen3.8-27B rather than an uncensored version<\/li>\n<\/ul>\n<h2>How It Differs from Similar Models<\/h2>\n<ul>\n<li><strong>Difference from general GGUF quantized versions (Q3_K_M, IQ3 families, etc.)<\/strong>: While the file sizes are similar, the execution engines are completely different. While GGUF runs out of the box in llama.cpp, Ollama, and LM Studio, this version does not run in them (see the next section). Its strengths lie in providing numerical measurements of deviation from BF16 and the ability to handle multiple concurrent requests using vLLM<\/li>\n<li><strong>Difference from other EXL3 quantized versions<\/strong>: The EXL3 format itself is the standard format for ExLlamaV3, and other versions around 3 bits could potentially be created. The differences in OrcaSAQ2 lie in searching for layer-wise bit allocation distributions and providing a plugin to run it with vLLM. However, numerical comparisons against other EXL3 versions under identical conditions are not provided<\/li>\n<li><strong>Difference from the base Qwen3.8-27B (BF16)<\/strong>: The required memory drops to under a quarter, but image input capability is lost, and token-level agreement is limited to 93.2%. The choice is between the original model if accuracy is the top priority, or this version if fitting into 16GB is the priority<\/li>\n<\/ul>\n<p><!-- lmw:hardware --><\/p>\n<h2>Hardware Requirements<\/h2>\n<p><strong>Estimated requirements (calculated by Local Model Watch)<\/strong> \u2014 27.8B parameters (taken from the base model Qwen\/Qwen3.8-27B)<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Your VRAM<\/th>\n<th>Quantization<\/th>\n<th>File size<\/th>\n<th>Est. memory needed<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>16GB (RTX 5060 Ti 16GB \/ 4060 Ti 16GB, etc.)<\/td>\n<td>EXL3<\/td>\n<td>11.4GB<\/td>\n<td>13.7GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model&#8217;s authors. Compare with other models in our <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-en\/\">VRAM quick reference<\/a>. What the quantization names mean: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">glossary<\/a>.<\/em><\/p>\n<p><!-- \/lmw:hardware --><\/p>\n<p><!-- lmw:runnability --><\/p>\n<h2>Can You Run It Locally?<\/h2>\n<p><strong>Not usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\n<p>The publisher ships EXL3 \/ safetensors only. Today it can be run with transformers, using the memory figures in the table above.<\/p>\n<p><strong>License \u2014 <code>apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\n<p><em>Compiled by this site&#8217;s code from the published formats, converted builds we have found, and each engine&#8217;s own model registry. &#8220;Not found&#8221; means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher&#8217;s original terms before relying on them.<\/em><\/p>\n<p><!-- \/lmw:runnability --><\/p>\n<p><!-- lmw:lab --><\/p>\n<h2>Our Own Measurements<\/h2>\n<p>Values we measured ourselves \u2014 not figures copied from the model card. We read this model&#8217;s files on our own server (no GPU), and because we cannot run this model on that server, we ran it on a rented cloud GPU (conditions below). How we measure, and the results for every model: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/observations-en\/\">Our Measurements<\/a>.<\/p>\n<h3>What We Found<\/h3>\n<p>OrcaSAQ-2-27B demonstrates high Japanese token efficiency, allowing more information to be packed into a limited context length. However, it showed instability in knowledge-based questions and long-form generation, such as truncated responses and repeated lines. Its Japanese token efficiency is 546 tokens per 1,000 characters, which is lower than the 688 tokens required by Qwen3, enabling more compact processing of Japanese text.<\/p>\n<p>Testing with the EXL3 3.21bit format on a cloud GPU (NVIDIA L4) showed that the model requires 13.7GB of VRAM for loading, with a peak usage of 14.0GB during generation. The generation speed was 17.19 tokens per second. In this environment, the model successfully followed instructions for tasks such as summarization, haiku creation, and math word problems, correctly identifying the answer of 280 yen for the latter.<\/p>\n<p>Conversely, the model hit the 1,024-token limit and truncated its response when asked to list prefectures without a coastline. During this process, a single line was repeated 11 times, and it only listed 4 out of the 8 correct prefectures while incorrectly including 2 coastal prefectures. Since repeated lines were also observed twice during the drafting of a business email, caution is advised when using this model for long-form generation or specific knowledge retrieval tasks.<\/p>\n<h3>Japanese Token Efficiency<\/h3>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Tokenizer<\/th>\n<th>Tokens per 1,000 Japanese characters<\/th>\n<th>Ratio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><strong>This model<\/strong><\/td>\n<td><strong>546<\/strong><\/td>\n<td><strong>0.99\u00d7<\/strong><\/td>\n<\/tr>\n<tr>\n<td>Qwen3<\/td>\n<td>688<\/td>\n<td>1.26\u00d7<\/td>\n<\/tr>\n<tr>\n<td>Llama 3.2<\/td>\n<td>744<\/td>\n<td>1.36\u00d7<\/td>\n<\/tr>\n<tr>\n<td>Gemma 3<\/td>\n<td>564<\/td>\n<td>1.03\u00d7<\/td>\n<\/tr>\n<tr>\n<td>gpt-oss<\/td>\n<td>795<\/td>\n<td>1.45\u00d7<\/td>\n<\/tr>\n<tr>\n<td>LLM-jp-3<\/td>\n<td>497<\/td>\n<td>0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>It needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\n<p>Counted with the <code>tokenizer.json<\/code> of <a href=\"https:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B\">orcarouter\/OrcaSAQ-2-27B<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<h3>Running the EXL3 Weights With ExLlamaV3 (Cloud GPU)<\/h3>\n<p>llama.cpp does not read the EXL3 format, so we ran this repository&#8217;s weights as published with ExLlamaV3, the open-source engine for EXL3, on a rented cloud GPU. We did not run any code from the publisher&#8217;s repository, and the run happened in a container with its network cut off.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Result<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Weight files<\/td>\n<td>11.4GB<\/td>\n<\/tr>\n<tr>\n<td>VRAM right after loading (KV cache for 16384 tokens included)<\/td>\n<td>13.7GB<\/td>\n<\/tr>\n<tr>\n<td>Peak VRAM while answering the five questions<\/td>\n<td>14.0GB<\/td>\n<\/tr>\n<tr>\n<td>Generation (median over the five answers, ExLlamaV3)<\/td>\n<td>17.2 tokens\/s<\/td>\n<\/tr>\n<tr>\n<td>Generation in Japanese characters<\/td>\n<td>about 31.5 characters\/s<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>Conditions: NVIDIA L4 (VRAM 22GB) on Modal, ExLlamaV3 1.5.3, PyTorch 2.8.0+cu128, <a href=\"https:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B\/tree\/15d20d7e9ae4fd89d1a47878f69381760169445b\">orcarouter\/OrcaSAQ-2-27B<\/a> at revision <code>15d20d7e9a<\/code> (EXL3 3.21bit, greedy decoding, thinking off), measured 2026-10-01 (JST).<\/p>\n<p>The speed is a reference for this one GPU and engine; it is not a speed you would get with llama.cpp or on a different GPU. The embedding layer in this repository is stored as int8 values plus a per-row scale, which ExLlamaV3 cannot read as is, so before loading we multiplied them back into a bf16 weight ourselves (the code is in the procedure below). The publisher has not documented this conversion, so this is our own reading of the format.<\/p>\n<h3>Answers to Our Japanese Questions (Verbatim)<\/h3>\n<p>We ask every model the same five questions in Japanese under the conditions above (temperature 0, thinking turned off where the chat template allows it, up to 1024 tokens) and publish the answers unedited. We do not grade them; where a question has a single correct answer, we show it for comparison.<\/p>\n<p><strong>Summarization<\/strong><\/p>\n<blockquote>\n<p>\u6b21\u306e\u8a18\u4e8b\u30923\u6587\u3067\u8981\u7d04\u3057\u3066\u304f\u3060\u3055\u3044\u3002<\/p>\n<p>\u307f\u3069\u308a\u5e02\u306f\u4eca\u5e74\u306e\u6625\u3001\u5e02\u5185\u306b\u4e09\u3064\u3042\u308b\u516c\u7acb\u56f3\u66f8\u9928\u306e\u3046\u3061\u6700\u3082\u53e4\u3044\u4e2d\u592e\u56f3\u66f8\u9928\u306e\u6539\u4fee\u3092\u7d42\u3048\u3001\u5341\u6708\u4e00\u65e5\u306b\u518d\u958b\u9928\u3057\u305f\u3002\u6539\u4fee\u306f\u7d04\u4e00\u5e74\u534a\u306b\u308f\u305f\u308a\u3001\u7dcf\u4e8b\u696d\u8cbb\u306f\u7d04\u5341\u4e8c\u5104\u5186\u3002\u3053\u306e\u3046\u3061\u7d04\u56db\u5104\u5186\u306f\u56fd\u306e\u88dc\u52a9\u91d1\u3067\u307e\u304b\u306a\u308f\u308c\u305f\u3002<\/p>\n<p>\u6700\u5927\u306e\u5909\u5316\u306f\u3001\u95b2\u89a7\u5e2d\u306e\u914d\u7f6e\u3067\u3042\u308b\u3002\u4ee5\u524d\u306f\u9928\u5185\u306e\u5ea7\u5e2d\u306e\u5927\u534a\u304c\u5b66\u7fd2\u7528\u306e\u673a\u3067\u3001\u8a66\u9a13\u671f\u9593\u306b\u306a\u308b\u3068\u4e2d\u9ad8\u751f\u3067\u57cb\u307e\u308a\u3001\u9ad8\u9f62\u306e\u5229\u7528\u8005\u3084\u5c0f\u3055\u306a\u5b50\u3069\u3082\u9023\u308c\u304c\u5ea7\u308c\u306a\u3044\u3068\u3044\u3046\u82e6\u60c5\u304c\u6bce\u5e74\u5bc4\u305b\u3089\u308c\u3066\u3044\u305f\u3002\u6539\u4fee\u5f8c\u306f\u5ea7\u5e2d\u6570\u3092\u767e\u4e8c\u5341\u5e2d\u304b\u3089\u4e8c\u767e\u5e2d\u306b\u5897\u3084\u3057\u305f\u3046\u3048\u3067\u3001\u5b66\u7fd2\u7528\u306e\u673a\u3092\u4e09\u968e\u306b\u307e\u3068\u3081\u3001\u4e00\u968e\u306f\u4f1a\u8a71\u304c\u3067\u304d\u308b\u300c\u306b\u304e\u308f\u3044\u30d5\u30ed\u30a2\u300d\u3001\u4e8c\u968e\u306f\u9759\u304b\u306b\u672c\u3092\u8aad\u3080\u300c\u95b2\u89a7\u30d5\u30ed\u30a2\u300d\u306b\u5206\u3051\u305f\u3002\u4e00\u968e\u306b\u306f\u4e88\u7d04\u5236\u306e\u30b0\u30eb\u30fc\u30d7\u5b66\u7fd2\u5ba4\u304c\u56db\u5ba4\u3068\u3001\u98f2\u307f\u7269\u3092\u6301\u3061\u8fbc\u3081\u308b\u30ab\u30d5\u30a7\u30b9\u30da\u30fc\u30b9\u3082\u8a2d\u3051\u3089\u308c\u305f\u3002<\/p>\n<p>\u8535\u66f8\u306e\u9762\u3067\u3082\u898b\u76f4\u3057\u304c\u9032\u3093\u3060\u3002\u8cb8\u51fa\u306e\u5c11\u306a\u3044\u53e4\u3044\u8cc7\u6599\u7d04\u4e09\u4e07\u518a\u3092\u5e02\u5185\u306e\u4ed6\u306e\u9928\u3084\u4fdd\u5b58\u66f8\u5eab\u306b\u79fb\u3057\u3001\u4ee3\u308f\u308a\u306b\u5b50\u3069\u3082\u5411\u3051\u306e\u7d75\u672c\u3068\u3001\u4ecb\u8b77\u30fb\u5b50\u80b2\u3066\u30fb\u4ed5\u4e8b\u63a2\u3057\u3068\u3044\u3063\u305f\u66ae\u3089\u3057\u306b\u95a2\u308f\u308b\u5b9f\u7528\u66f8\u3092\u7d04\u4e00\u4e07\u4e94\u5343\u518a\u5897\u3084\u3057\u305f\u3002\u96fb\u5b50\u66f8\u7c4d\u306e\u8cb8\u51fa\u30b5\u30fc\u30d3\u30b9\u3082\u59cb\u307e\u308a\u3001\u5229\u7528\u767b\u9332\u3092\u3059\u308c\u3070\u81ea\u5b85\u304b\u3089\u4e00\u5ea6\u306b\u4e09\u518a\u307e\u3067\u3001\u4e8c\u9031\u9593\u501f\u308a\u3089\u308c\u308b\u3002<\/p>\n<p>\u5efa\u7269\u305d\u306e\u3082\u306e\u306b\u3082\u624b\u304c\u5165\u3063\u305f\u3002\u7bc9\u56db\u5341\u5e74\u3092\u8d85\u3048\u3066\u3044\u305f\u7a7a\u8abf\u8a2d\u5099\u3092\u5165\u308c\u66ff\u3048\u3001\u5c4b\u4e0a\u306b\u306f\u592a\u967d\u5149\u30d1\u30cd\u30eb\u3092\u8a2d\u7f6e\u3057\u3066\u3001\u9928\u5185\u3067\u4f7f\u3046\u96fb\u529b\u306e\u7d04\u4e8c\u5272\u3092\u307e\u304b\u306a\u3048\u308b\u3088\u3046\u306b\u3057\u305f\u3002\u8eca\u3044\u3059\u306e\u5229\u7528\u8005\u304b\u3089\u8981\u671b\u306e\u591a\u304b\u3063\u305f\u30a8\u30ec\u30d9\u30fc\u30bf\u30fc\u306f\u4e00\u57fa\u304b\u3089\u4e8c\u57fa\u306b\u5897\u3048\u3001\u66f8\u67b6\u306e\u9593\u306e\u901a\u8def\u3082\u4ee5\u524d\u3088\u308a\u4e09\u5341\u30bb\u30f3\u30c1\u5e83\u3052\u3089\u308c\u305f\u3002\u5165\u53e3\u306b\u306f\u6bb5\u5dee\u306e\u306a\u3044\u30b9\u30ed\u30fc\u30d7\u304c\u65b0\u305f\u306b\u8a2d\u3051\u3089\u308c\u3001\u6388\u4e73\u5ba4\u3068\u591a\u76ee\u7684\u30c8\u30a4\u30ec\u3082\u5404\u968e\u306b\u4e00\u3064\u305a\u3064\u7f6e\u304b\u308c\u305f\u3002<\/p>\n<p>\u4e00\u65b9\u3067\u8ab2\u984c\u3082\u6b8b\u308b\u3002\u99d0\u8eca\u5834\u306f\u6539\u4fee\u524d\u3068\u540c\u3058\u56db\u5341\u53f0\u306e\u307e\u307e\u3067\u3001\u9031\u672b\u306b\u306f\u8fd1\u304f\u306e\u9053\u8def\u306b\u9806\u756a\u5f85\u3061\u306e\u8eca\u304c\u4e26\u3093\u3060\u3002\u5e02\u306f\u96a3\u63a5\u3059\u308b\u5e02\u6709\u5730\u3092\u81e8\u6642\u99d0\u8eca\u5834\u3068\u3057\u3066\u958b\u653e\u3057\u305f\u304c\u3001\u6052\u4e45\u7684\u306a\u5bfe\u7b56\u306f\u6c7a\u307e\u3063\u3066\u3044\u306a\u3044\u3002\u307e\u305f\u3001\u958b\u9928\u6642\u9593\u3092\u5e73\u65e5\u306e\u591c\u4e5d\u6642\u307e\u3067\u5ef6\u3070\u3057\u305f\u3053\u3068\u3067\u8077\u54e1\u306e\u8ca0\u62c5\u304c\u5897\u3048\u3066\u304a\u308a\u3001\u8077\u54e1\u56e3\u4f53\u306f\u4eba\u54e1\u306e\u8ffd\u52a0\u3092\u6c42\u3081\u3066\u3044\u308b\u3002<\/p>\n<p>\u5229\u7528\u8005\u306e\u53d7\u3051\u6b62\u3081\u306f\u304a\u304a\u3080\u306d\u597d\u610f\u7684\u3060\u3002\u9031\u306b\u4e8c\u5ea6\u3001\u5b6b\u3092\u9023\u308c\u3066\u6765\u308b\u3068\u3044\u3046\u4e03\u5341\u4ee3\u306e\u5973\u6027\u306f\u300c\u524d\u306f\u5b50\u3069\u3082\u304c\u58f0\u3092\u51fa\u3059\u3068\u5468\u308a\u306e\u76ee\u304c\u6c17\u306b\u306a\u3063\u305f\u304c\u3001\u4e00\u968e\u306a\u3089\u6c17\u517c\u306d\u306a\u304f\u7d75\u672c\u3092\u8aad\u3093\u3067\u3042\u3052\u3089\u308c\u308b\u300d\u3068\u8a71\u3059\u3002\u4e00\u65b9\u3001\u53d7\u9a13\u3092\u63a7\u3048\u305f\u9ad8\u6821\u751f\u306e\u4e2d\u306b\u306f\u300c\u5b66\u7fd2\u5e2d\u304c\u4e09\u968e\u3060\u3051\u306b\u306a\u308a\u3001\u5915\u65b9\u306f\u3059\u3050\u306b\u6e80\u5e2d\u306b\u306a\u308b\u300d\u3068\u4e0d\u6e80\u3092\u6f0f\u3089\u3059\u4eba\u3082\u3044\u308b\u3002\u56f3\u66f8\u9928\u306f\u3001\u5b66\u7fd2\u5e2d\u306e\u7a7a\u304d\u72b6\u6cc1\u3092\u30db\u30fc\u30e0\u30da\u30fc\u30b8\u3067\u78ba\u8a8d\u3067\u304d\u308b\u4ed5\u7d44\u307f\u3092\u5e74\u5185\u306b\u5c0e\u5165\u3059\u308b\u4e88\u5b9a\u3060\u3002<\/p>\n<p>\u518d\u958b\u9928\u306b\u5408\u308f\u305b\u3066\u3001\u50ac\u3057\u3082\u5897\u3048\u305f\u3002\u6bce\u9031\u571f\u66dc\u306e\u5348\u524d\u306b\u306f\u8aad\u307f\u805e\u304b\u305b\u4f1a\u3001\u6708\u306b\u4e00\u5ea6\u306f\u5730\u5143\u306e\u8fb2\u5bb6\u3084\u5546\u5e97\u4e3b\u3092\u8b1b\u5e2b\u306b\u62db\u304f\u300c\u307e\u3061\u306e\u5148\u751f\u300d\u8b1b\u5ea7\u304c\u958b\u304b\u308c\u308b\u3002\u5341\u4e00\u6708\u306b\u306f\u3001\u5e02\u5185\u306e\u4e2d\u5b66\u751f\u304c\u9078\u3093\u3060\u672c\u3092\u7d39\u4ecb\u3059\u308b\u5c55\u793a\u3082\u4e88\u5b9a\u3055\u308c\u3066\u3044\u308b\u3002<\/p>\n<p>\u9928\u9577\u306e\u5c71\u672c\u548c\u5b50\u3055\u3093\u306f\u300c\u672c\u3092\u501f\u308a\u308b\u5834\u6240\u304b\u3089\u3001\u4eba\u304c\u96c6\u307e\u3063\u3066\u904e\u3054\u3059\u5834\u6240\u3078\u5909\u3048\u305f\u304b\u3063\u305f\u3002\u518d\u958b\u9928\u304b\u3089\u306e\u4e00\u304b\u6708\u3067\u6765\u9928\u8005\u306f\u4ee5\u524d\u306e\u4e00\u30fb\u516b\u500d\u306b\u306a\u3063\u305f\u300d\u3068\u8a71\u3059\u3002\u5e02\u306f\u6765\u5e74\u5ea6\u3001\u6b8b\u308b\u4e8c\u3064\u306e\u56f3\u66f8\u9928\u306b\u3064\u3044\u3066\u3082\u5229\u7528\u8005\u3078\u306e\u805e\u304d\u53d6\u308a\u3092\u884c\u3044\u3001\u6539\u4fee\u306e\u5fc5\u8981\u6027\u3092\u691c\u8a0e\u3059\u308b\u65b9\u91dd\u3060\u3002<\/p>\n<\/blockquote>\n<p><em>(Summarize the following article in three sentences. (The article, about 1,200 Japanese characters, describes the renovation and reopening of a city&#8217;s central library: the budget, the new floor layout and seating, changes to the collection and e-book lending, building and accessibility work, remaining issues such as parking and staff workload, users&#8217; reactions, new events, and the city&#8217;s next steps.))<\/em><\/p>\n<pre><code class=\"language-text\">\u307f\u3069\u308a\u5e02\u4e2d\u592e\u56f3\u66f8\u9928\u306f\u7d0412\u5104\u5186\u306e\u6539\u4fee\u3092\u7d4c\u3066\u518d\u958b\u3057\u3001\u30d5\u30ed\u30a2\u3054\u3068\u306e\u6a5f\u80fd\u5206\u5316\u3084\u8535\u66f8\u306e\u898b\u76f4\u3057\u3001\u30d0\u30ea\u30a2\u30d5\u30ea\u30fc\u5bfe\u5fdc\u306a\u3069\u3092\u901a\u3058\u3066\u3001\u5358\u306a\u308b\u95b2\u89a7\u65bd\u8a2d\u304b\u3089\u5730\u57df\u4ea4\u6d41\u306e\u5834\u3078\u3068\u8ee2\u63db\u3057\u305f\u3002\u305d\u306e\u7d50\u679c\u3001\u6765\u9928\u8005\u306f\u4ee5\u524d\u6bd41.8\u500d\u306b\u5897\u52a0\u3057\u3001\u591a\u69d8\u306a\u5229\u7528\u8005\u306e\u30cb\u30fc\u30ba\u306b\u5fdc\u3048\u308b\u74b0\u5883\u304c\u6574\u3063\u305f\u4e00\u65b9\u3001\u99d0\u8eca\u5834\u4e0d\u8db3\u3084\u8077\u54e1\u8ca0\u62c5\u5897\u3068\u3044\u3063\u305f\u8ab2\u984c\u3082\u6b8b\u3063\u3066\u3044\u308b\u3002\u5e02\u306f\u4eca\u5f8c\u306e\u5229\u7528\u8005\u8abf\u67fb\u3092\u901a\u3058\u3066\u6b8b\u308b2\u9928\u306e\u6539\u4fee\u5fc5\u8981\u6027\u3092\u691c\u8a0e\u3057\u3001\u56f3\u66f8\u9928\u6a5f\u80fd\u306e\u3055\u3089\u306a\u308b\u5145\u5b9f\u3092\u56f3\u308b\u65b9\u91dd\u3092\u793a\u3057\u3066\u3044\u308b\u3002\n<\/code><\/pre>\n<p><strong>Formal business email<\/strong><\/p>\n<blockquote>\n<p>\u53d6\u5f15\u5148\u306e\u62c5\u5f53\u8005\u306b\u3001\u6765\u9031\u706b\u66dc\u65e5\u306e\u6253\u3061\u5408\u308f\u305b\u3092\u6c34\u66dc\u65e5\u306b\u5909\u66f4\u3057\u3066\u307b\u3057\u3044\u3068\u304a\u9858\u3044\u3059\u308b\u30e1\u30fc\u30eb\u3092\u3001\u4e01\u5be7\u306a\u656c\u8a9e\u3067\u66f8\u3044\u3066\u304f\u3060\u3055\u3044\u3002<\/p>\n<\/blockquote>\n<p><em>(Write an email in polite Japanese (keigo) asking a client&#8217;s contact person to move next Tuesday&#8217;s meeting to Wednesday.)<\/em><\/p>\n<pre><code class=\"language-text\">\u4ef6\u540d\uff1a\u3010\u3054\u76f8\u8ac7\u3011\u6765\u9031\u706b\u66dc\u65e5\u306e\u6253\u3061\u5408\u308f\u305b\u65e5\u7a0b\u5909\u66f4\u306e\u304a\u9858\u3044\n\n\u25cb\u25cb\u682a\u5f0f\u4f1a\u793e\n\u25cb\u25cb\u90e8 \u25cb\u25cb\u69d8\n\n\u3044\u3064\u3082\u5927\u5909\u304a\u4e16\u8a71\u306b\u306a\u3063\u3066\u304a\u308a\u307e\u3059\u3002\n\u682a\u5f0f\u4f1a\u793e\u25b3\u25b3\u306e\u25b3\u25b3\u3067\u3059\u3002\n\n\u7a81\u7136\u306e\u3054\u9023\u7d61\u3068\u306a\u308a\u3001\u5927\u5909\u5931\u793c\u3044\u305f\u3057\u307e\u3059\u3002\n\n\u6765\u9031\u706b\u66dc\u65e5\uff08\u25cb\u6708\u25cb\u65e5\uff09\u306b\u4e88\u5b9a\u3057\u3066\u304a\u308a\u307e\u3059\u6253\u3061\u5408\u308f\u305b\u306e\u4ef6\u3067\u3001\u8aa0\u306b\u6050\u308c\u5165\u308a\u307e\u3059\u304c\u3001\u65e5\u7a0b\u306e\u5909\u66f4\u3092\u304a\u9858\u3044\u3057\u305f\u304f\u3054\u9023\u7d61\u3044\u305f\u3057\u307e\u3057\u305f\u3002\n\n\u793e\u5185\u4e8b\u60c5\u306b\u3088\u308a\u3001\u5f53\u65e5\u306e\u53c2\u52a0\u4e88\u5b9a\u8005\u304c\u90fd\u5408\u3064\u304b\u306a\u3044\u72b6\u6cc1\u3068\u306a\u308a\u307e\u3057\u305f\u305f\u3081\u3001\u3082\u3057\u53ef\u80fd\u3067\u3042\u308c\u3070\u3001**\u7fcc\u65e5\u306e\u6c34\u66dc\u65e5\uff08\u25cb\u6708\u25cb\u65e5\uff09**\u306b\u5909\u66f4\u3055\u305b\u3066\u3044\u305f\u3060\u304f\u3053\u3068\u306f\u53ef\u80fd\u3067\u3057\u3087\u3046\u304b\u3002\n\n\u3064\u304d\u307e\u3057\u3066\u306f\u3001\u4ee5\u4e0b\u306e\u65e5\u7a0b\u3067\u6539\u3081\u3066\u3054\u8abf\u6574\u3044\u305f\u3060\u3051\u307e\u3059\u3068\u5e78\u3044\u3067\u3059\u3002\n\n\u25a0\u5e0c\u671b\u65e5\u7a0b\n\u25cb\u6708\u25cb\u65e5\uff08\u6c34\uff09 10:00\u301c12:00\n\u203b\u4e0a\u8a18\u306e\u6642\u9593\u5e2f\u3067\u8abf\u6574\u304c\u96e3\u3057\u3044\u5834\u5408\u306f\u3001\u6c34\u66dc\u65e5\u306e\u4ed6\u306e\u6642\u9593\u5e2f\u3084\u3001\u4ee5\u964d\u306e\u65e5\u7a0b\u3067\u3082\u3054\u76f8\u8ac7\u306b\u5fdc\u3058\u307e\u3059\u3002\n\n\u3054\u591a\u5fd9\u306e\u3068\u3053\u308d\u3001\u6025\u306a\u304a\u9858\u3044\u3068\u306a\u308a\u5927\u5909\u7533\u3057\u8a33\u3054\u3056\u3044\u307e\u305b\u3093\u3002\n\u3054\u78ba\u8a8d\u306e\u307b\u3069\u3001\u4f55\u5352\u3088\u308d\u3057\u304f\u304a\u9858\u3044\u7533\u3057\u4e0a\u3052\u307e\u3059\u3002\n\n\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\n\u682a\u5f0f\u4f1a\u793e\u25b3\u25b3\n\u25b3\u25b3\u90e8 \u25b3\u25b3\uff08\u6c0f\u540d\uff09\nTEL: 03-XXXX-XXXX\nMail: xxxxx@xxxx.com\n\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\n<\/code><\/pre>\n<p><strong>Word problem<\/strong><\/p>\n<blockquote>\n<p>\u308a\u3093\u3054\u304c1\u500b120\u5186\u3001\u307f\u304b\u3093\u304c1\u500b45\u5186\u3067\u3059\u3002\u308a\u3093\u3054\u30923\u500b\u3068\u307f\u304b\u3093\u30928\u500b\u8cb7\u3063\u3066\u30011,000\u5186\u672d\u3067\u6255\u3044\u307e\u3057\u305f\u3002\u304a\u3064\u308a\u306f\u3044\u304f\u3089\u3067\u3059\u304b\u3002\u8003\u3048\u65b9\u3082\u793a\u3057\u3066\u304f\u3060\u3055\u3044\u3002<\/p>\n<\/blockquote>\n<p><em>(Apples cost 120 yen each and mandarins cost 45 yen each. You buy 3 apples and 8 mandarins and pay with a 1,000-yen bill. How much change do you get? Show your reasoning.)<\/em><\/p>\n<pre><code class=\"language-text\">\u304a\u3064\u308a\u306e\u91d1\u984d\u3092\u6c42\u3081\u308b\u305f\u3081\u306b\u3001\u307e\u305a\u308a\u3093\u3054\u3068\u307f\u304b\u3093\u306e\u5408\u8a08\u4ee3\u91d1\u3092\u8a08\u7b97\u3057\u3001\u305d\u3053\u304b\u30891,000\u5186\u3092\u5f15\u304f\u624b\u9806\u3067\u8003\u3048\u307e\u3059\u3002\n\n### 1. \u8003\u3048\u65b9\u306e\u624b\u9806\n\n1.  **\u308a\u3093\u3054\u306e\u4ee3\u91d1**\u3092\u8a08\u7b97\u3059\u308b\u3002\n    *   \u5358\u4fa1 \u00d7 \u500b\u6570\n2.  **\u307f\u304b\u3093\u306e\u4ee3\u91d1**\u3092\u8a08\u7b97\u3059\u308b\u3002\n    *   \u5358\u4fa1 \u00d7 \u500b\u6570\n3.  **\u5408\u8a08\u4ee3\u91d1**\u3092\u8a08\u7b97\u3059\u308b\u3002\n    *   \u308a\u3093\u3054\u306e\u4ee3\u91d1 + \u307f\u304b\u3093\u306e\u4ee3\u91d1\n4.  **\u304a\u3064\u308a**\u3092\u8a08\u7b97\u3059\u308b\u3002\n    *   \u652f\u6255\u3063\u305f\u91d1\u984d\uff081,000\u5186\uff09 - \u5408\u8a08\u4ee3\u91d1\n\n---\n\n### 2. \u8a08\u7b97\u904e\u7a0b\n\n**\u2460 \u308a\u3093\u3054\u306e\u4ee3\u91d1**\n$$ 120 \\text{\u5186} \\times 3 = 360 \\text{\u5186} $$\n\n**\u2461 \u307f\u304b\u3093\u306e\u4ee3\u91d1**\n$$ 45 \\text{\u5186} \\times 8 = 360 \\text{\u5186} $$\n\n**\u2462 \u5408\u8a08\u4ee3\u91d1**\n$$ 360 \\text{\u5186} + 360 \\text{\u5186} = 720 \\text{\u5186} $$\n\n**\u2463 \u304a\u3064\u308a**\n$$ 1,000 \\text{\u5186} - 720 \\text{\u5186} = 280 \\text{\u5186} $$\n\n---\n\n### 3. \u7b54\u3048\n\n\u304a\u3064\u308a\u306f **280\u5186** \u3067\u3059\u3002\n<\/code><\/pre>\n<p>Reference: The correct answer is 280 yen (120\u00d73 = 360, 45\u00d78 = 360, 1,000 \u2212 720 = 280).<\/p>\n<p><strong>Knowledge<\/strong><\/p>\n<blockquote>\n<p>\u65e5\u672c\u306e\u90fd\u9053\u5e9c\u770c\u306e\u3046\u3061\u3001\u6d77\u306b\u9762\u3057\u3066\u3044\u306a\u3044\u770c\u3092\u3059\u3079\u3066\u6319\u3052\u3066\u304f\u3060\u3055\u3044\u3002<\/p>\n<\/blockquote>\n<p><em>(List all of Japan&#8217;s prefectures that do not border the sea.)<\/em><\/p>\n<pre><code class=\"language-text\">\u65e5\u672c\u306e\u90fd\u9053\u5e9c\u770c\u306e\u4e2d\u3067\u3001\u6d77\u306b\u9762\u3057\u3066\u3044\u306a\u3044\uff08\u5185\u9678\u770c\uff09\u306e\u306f\u4ee5\u4e0b\u306e**9\u770c**\u3067\u3059\u3002\n\n1. **\u5c71\u68a8\u770c**\n2. **\u9577\u91ce\u770c**\n3. **\u5c90\u961c\u770c**\n4. **\u4e09\u91cd\u770c**\uff08\u203b\u6ce8\uff1a\u4e09\u91cd\u770c\u306f\u4f0a\u52e2\u6e7e\u306b\u9762\u3057\u3066\u3044\u308b\u305f\u3081\u3001\u53b3\u5bc6\u306b\u306f\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\u3057\u304b\u3057\u3001\u591a\u304f\u306e\u6587\u8108\u3067\u300c\u5185\u9678\u770c\u300d\u3068\u3057\u3066\u6271\u308f\u308c\u308b\u3053\u3068\u304c\u3042\u308a\u307e\u3059\u304c\u3001\u5730\u7406\u7684\u306b\u306f\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\u3053\u3053\u3067\u306f**\u6d77\u306b\u9762\u3057\u3066\u3044\u306a\u3044**\u3068\u3044\u3046\u53b3\u5bc6\u306a\u5b9a\u7fa9\u306b\u57fa\u3065\u304d\u3001\u4e09\u91cd\u770c\u306f\u9664\u5916\u3057\u307e\u3059\u3002\uff09\n\n\u203b\u8a02\u6b63\uff1a\u4e09\u91cd\u770c\u306f\u4f0a\u52e2\u6e7e\u306b\u9762\u3057\u3066\u3044\u308b\u305f\u3081\u3001\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\u3057\u305f\u304c\u3063\u3066\u3001\u5185\u9678\u770c\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002\n\n\u6b63\u3057\u3044\u5185\u9678\u770c\uff08\u6d77\u306b\u9762\u3057\u3066\u3044\u306a\u3044\u770c\uff09\u306f\u4ee5\u4e0b\u306e**9\u770c**\u3067\u3059\u3002\n\n1. **\u5c71\u68a8\u770c**\n2. **\u9577\u91ce\u770c**\n3. **\u5c90\u961c\u770c**\n4. **\u6ecb\u8cc0\u770c**\uff08\u7435\u7436\u6e56\u306f\u6e56\u3067\u3042\u308a\u3001\u6d77\u3067\u306f\u306a\u3044\u305f\u3081\u5185\u9678\u770c\u3068\u3055\u308c\u307e\u3059\uff09\n5. **\u611b\u77e5\u770c**\uff08\u203b\u6ce8\uff1a\u611b\u77e5\u770c\u306f\u592a\u5e73\u6d0b\u3068\u4f0a\u52e2\u6e7e\u306b\u9762\u3057\u3066\u3044\u308b\u305f\u3081\u3001\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\u5185\u9678\u770c\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002\uff09\n\n\u203b\u518d\u5ea6\u78ba\u8a8d\uff1a\u611b\u77e5\u770c\u306f\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\n\n\u6b63\u3057\u3044\u5185\u9678\u770c\uff08\u6d77\u306b\u9762\u3057\u3066\u3044\u306a\u3044\u770c\uff09\u306f\u4ee5\u4e0b\u306e**9\u770c**\u3067\u3059\u3002\n\n1. **\u5c71\u68a8\u770c**\n2. **\u9577\u91ce\u770c**\n3. **\u5c90\u961c\u770c**\n4. **\u6ecb\u8cc0\u770c**\n5. **\u4e09\u91cd\u770c**\uff08\u203b\u4f0a\u52e2\u6e7e\u306b\u9762\u3057\u3066\u3044\u308b\u305f\u3081\u3001\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\u5185\u9678\u770c\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002\uff09\n\n\u203b\u518d\u5ea6\u78ba\u8a8d\uff1a\u4e09\u91cd\u770c\u306f\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\n\n\u6b63\u3057\u3044\u5185\u9678\u770c\uff08\u6d77\u306b\u9762\u3057\u3066\u3044\u306a\u3044\u770c\uff09\u306f\u4ee5\u4e0b\u306e**9\u770c**\u3067\u3059\u3002\n\n1. **\u5c71\u68a8\u770c**\n2. **\u9577\u91ce\u770c**\n3. **\u5c90\u961c\u770c**\n4. **\u6ecb\u8cc0\u770c**\n5. **\u611b\u77e5\u770c**\uff08\u203b\u592a\u5e73\u6d0b\u3068\u4f0a\u52e2\u6e7e\u306b\u9762\u3057\u3066\u3044\u308b\u305f\u3081\u3001\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\u5185\u9678\u770c\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002\uff09\n\n\u203b\u518d\u5ea6\u78ba\u8a8d\uff1a\u611b\u77e5\u770c\u306f\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\n\n\u6b63\u3057\u3044\u5185\u9678\u770c\uff08\u6d77\u306b\u9762\u3057\u3066\u3044\u306a\u3044\u770c\uff09\u306f\u4ee5\u4e0b\u306e**9\u770c**\u3067\u3059\u3002\n\n1. **\u5c71\u68a8\u770c**\n2. **\u9577\u91ce\u770c**\n3. **\u5c90\u961c\u770c**\n4. **\u6ecb\u8cc0\u770c**\n5. **\u4e09\u91cd\u770c**\uff08\u203b\u4f0a\u52e2\u6e7e\u306b\u9762\u3057\u3066\u3044\u308b\u305f\u3081\u3001\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\u5185\u9678\u770c\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002\uff09\n\n\u203b\u518d\u5ea6\u78ba\u8a8d\uff1a\u4e09\u91cd\u770c\u306f\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\n\n\u6b63\u3057\u3044\u5185\u9678\u770c\uff08\u6d77\u306b\u9762\u3057\u3066\u3044\u306a\u3044\u770c\uff09\u306f\u4ee5\u4e0b\u306e**9\u770c**\u3067\u3059\u3002\n\n1. **\u5c71\u68a8\u770c**\n2. **\u9577\u91ce\u770c**\n3. **\u5c90\u961c\u770c**\n4. **\u6ecb\u8cc0\u770c**\n5. **\u611b\u77e5\u770c**\uff08\u203b\u592a\u5e73\u6d0b\u3068\u4f0a\u52e2\u6e7e\u306b\u9762\u3057\u3066\u3044\u308b\u305f\u3081\u3001\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\u5185\u9678\u770c\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002\uff09\n\n\u203b\u518d\u5ea6\u78ba\u8a8d\uff1a\u611b\u77e5\u770c\u306f\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\n\n\u6b63\u3057\u3044\u5185\u9678\u770c\uff08\u6d77\u306b\u9762\u3057\u3066\u3044\u306a\u3044\u770c\uff09\u306f\u4ee5\u4e0b\u306e**9\u770c**\u3067\u3059\u3002\n\n1. **\u5c71\u68a8\u770c**\n2. **\u9577\u91ce\u770c**\n3. **\u5c90\u961c\u770c**\n4. **\u6ecb\u8cc0\u770c**\n5. **\u4e09\u91cd\u770c**\uff08\u203b\u4f0a\u52e2\u6e7e\u306b\u9762\u3057\u3066\u3044\u308b\u305f\u3081\u3001\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\u5185\u9678\u770c\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002\uff09\n\n\u203b\u518d\u5ea6\u78ba\u8a8d\uff1a\u4e09\u91cd\u770c\u306f\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\n\n\u6b63\u3057\u3044\u5185\u9678\u770c\uff08\u6d77\u306b\u9762\u3057\u3066\u3044\u306a\u3044\u770c\uff09\u306f\u4ee5\u4e0b\u306e**9\u770c**\u3067\u3059\u3002\n\n1. **\u5c71\u68a8\u770c**\n2. **\u9577\u91ce\u770c**\n3. **\u5c90\u961c\u770c**\n4. **\u6ecb\u8cc0\u770c**\n5. **\u611b\u77e5\u770c**\uff08\u203b\u592a\u5e73\u6d0b\u3068\u4f0a\u52e2\u6e7e\u306b\u9762\u3057\u3066\u3044\u308b\u305f\u3081\u3001\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\u5185\u9678\u770c\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002\uff09\n\n\u203b\u518d\u5ea6\u78ba\u8a8d\uff1a\u611b\u77e5\u770c\u306f\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\n\n\u6b63\u3057\u3044\u5185\u9678\u770c\uff08\u6d77\u306b\u9762\u3057\u3066\u3044\u306a\u3044\u770c\uff09\u306f\u4ee5\u4e0b\u306e**9\u770c**\u3067\u3059\u3002\n\n1. **\u5c71\u68a8\u770c**\n2. **\u9577\u91ce\u770c**\n3. **\u5c90\u961c\u770c**\n4. **\u6ecb\u8cc0\u770c**\n5. **\u4e09\u91cd\u770c**\uff08\u203b\u4f0a\u52e2\u6e7e\u306b\u9762\u3057\u3066\u3044\u308b\u305f\u3081\u3001\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\u5185\u9678\u770c\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002\uff09\n\n\u203b\u518d\u5ea6\u78ba\u8a8d\uff1a\u4e09\u91cd\u770c\u306f\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\n\n\u6b63\u3057\u3044\u5185\u9678\u770c\uff08\u6d77\u306b\u9762\u3057\u3066\u3044\u306a\u3044\u770c\uff09\u306f\u4ee5\u4e0b\u306e**9\u770c**\u3067\u3059\u3002\n\n1. **\u5c71\u68a8\u770c**\n2. **\u9577\u91ce\u770c**\n3. **\u5c90\u961c\u770c**\n4. **\u6ecb\u8cc0\u770c**\n5. **\u611b\u77e5\u770c**\uff08\u203b\u592a\u5e73\u6d0b\u3068\u4f0a\u52e2\u6e7e\u306b\u9762\u3057\u3066\u3044\u308b\u305f\u3081\u3001\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\u5185\u9678\u770c\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002\uff09\n\n\u203b\u518d\u5ea6\u78ba\u8a8d\uff1a\u611b\u77e5\u770c\u306f\u6d77\u306b\u9762\u3057\u3066\u3044\u307e\u3059\u3002\n\n\u6b63\u3057\u3044\u5185\u9678\u770c\uff08\u6d77\u306b\u9762\u3057\u3066\u3044\u306a\u3044\u770c\uff09\u306f\u4ee5\u4e0b\u306e**9\u770c**\u3067\u3059\u3002\n\n1. **\u5c71\u68a8\u770c**\n2. **\u9577\u91ce\u770c\n<\/code><\/pre>\n<p><em>(Cut off at the 1024-token limit.)<\/em><\/p>\n<p>Reference: The correct answer is eight prefectures: Tochigi, Gunma, Saitama, Yamanashi, Nagano, Gifu, Shiga and Nara.<\/p>\n<p><strong>Constrained writing<\/strong><\/p>\n<blockquote>\n<p>\u300c\u6625\u300d\u3092\u30c6\u30fc\u30de\u306b\u3057\u305f\u4ff3\u53e5\u30923\u3064\u3001\u4e94\u30fb\u4e03\u30fb\u4e94\u306e\u97f3\u6570\u3092\u5b88\u3063\u3066\u4f5c\u3063\u3066\u304f\u3060\u3055\u3044\u3002\u305d\u308c\u305e\u308c\u306b\u4e00\u8a00\u306e\u8aac\u660e\u3092\u4ed8\u3051\u3066\u304f\u3060\u3055\u3044\u3002<\/p>\n<\/blockquote>\n<p><em>(Write three haiku on the theme of spring, keeping the 5-7-5 syllable pattern, each with a one-line explanation.)<\/em><\/p>\n<pre><code class=\"language-text\">\u3054\u6307\u5b9a\u306e\u6761\u4ef6\u306b\u57fa\u3065\u304d\u3001\u300c\u6625\u300d\u3092\u30c6\u30fc\u30de\u306b\u3057\u305f\u4ff3\u53e5\u30923\u3064\u4f5c\u6210\u3057\u307e\u3057\u305f\u3002\u305d\u308c\u305e\u308c\u306b\u4e00\u8a00\u306e\u89e3\u8aac\u3092\u6dfb\u3048\u3066\u3044\u307e\u3059\u3002\n\n1. **\u82b1\u5439\u96ea \u821e\u3044\u6563\u308b\u7a7a\u306b \u6625\u306e\u98a8**\n   &gt; \u89e3\u8aac\uff1a\u685c\u306e\u82b1\u3073\u3089\u304c\u821e\u3044\u6563\u308b\u60c5\u666f\u3068\u3001\u305d\u306e\u4e2d\u3092\u629c\u3051\u308b\u67d4\u3089\u304b\u306a\u98a8\u3067\u3001\u6625\u306e\u79fb\u308d\u3044\u3092\u8868\u73fe\u3057\u307e\u3057\u305f\u3002\n\n2. **\u65b0\u7dd1\u306e \u6728\u6f0f\u308c\u65e5\u63fa\u308c\u3066 \u6625\u306e\u5149**\n   &gt; \u89e3\u8aac\uff1a\u82e5\u8449\u304c\u8302\u308a\u59cb\u3081\u305f\u6728\u3005\u304b\u3089\u5dee\u3057\u8fbc\u3080\u3001\u63fa\u308c\u308b\u5149\u306e\u7f8e\u3057\u3055\u3092\u6349\u3048\u307e\u3057\u305f\u3002\n\n3. **\u86d9\u9cf4\u304f \u7530\u3093\u307c\u306e\u6625\u306e \u671d\u306e\u7a7a\u6c17**\n   &gt; \u89e3\u8aac\uff1a\u7530\u3093\u307c\u3067\u9cf4\u304d\u59cb\u3081\u308b\u86d9\u306e\u58f0\u3068\u3001\u6e05\u3005\u3057\u3044\u671d\u306e\u7a7a\u6c17\u611f\u3067\u3001\u6625\u306e\u751f\u547d\u529b\u3092\u8868\u73fe\u3057\u307e\u3057\u305f\u3002\n<\/code><\/pre>\n<h4>Setup, Download and Run Steps (What We Actually Ran)<\/h4>\n<p>We split the work into two containers on Modal. The downloading container is a Debian (slim) image with Python 3.12; it only fetches files. The running container is <code>nvidia\/cuda:12.8.1-devel-ubuntu22.04<\/code> with Python 3.12 and has no network access. ExLlamaV3&#8217;s CUDA extension is compiled while the image is built, so no compiling happens on the GPU. These are the commands that built the two environments.<\/p>\n<pre><code class=\"language-bash\"># 1. Environment for downloading (no GPU)\npython -m pip install huggingface_hub\n\n# 2. Environment for running (GPU, network blocked)\nexport TORCH_CUDA_ARCH_LIST=&quot;8.0;8.9&quot; MAX_JOBS=&quot;4&quot;\npip install torch==2.8.0 --index-url https:\/\/download.pytorch.org\/whl\/cu128\npip install transformers jinja2 ninja setuptools wheel\npip install --no-build-isolation exllamav3==1.5.3\n<\/code><\/pre>\n<p>The first container ran <code>download(job)<\/code> from <code>remote_code_remote.py<\/code> and the second ran <code>run(job)<\/code> from <code>exl3_remote.py<\/code> (both written by us; the second in full below). Files go under <code>\/models\/&lt;dir&gt;<\/code>:<\/p>\n<pre><code class=\"language-python\">&quot;&quot;&quot;\nModal \u306e\u30b3\u30f3\u30c6\u30ca\u306e\u4e2d\u3067\u52d5\u304f\u3001ExLlamaV3 \u306b\u3088\u308b EXL3 \u5f62\u5f0f\u306e\u30e2\u30c7\u30eb\u306e\u5b9f\u6e2c(2026-09-30)\u3002\nlmw\/lab\/gpu_modal.py \u306e run_exl3 \u304b\u3089\u547c\u3070\u308c\u308b\u3002\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u306f remote_code_remote.download \u3092\u4f7f\u3046\n(\u91cd\u307f\u306f safetensors \u3060\u3051\u30fb\u914d\u5e03\u5143\u306e\u30b3\u30fc\u30c9\u306f\u843d\u3068\u3055\u305a\u5b9f\u884c\u3082\u3057\u306a\u3044)\u3002\n\n\u5b9f\u884c\u3059\u308b\u30b3\u30f3\u30c6\u30ca\u306f\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u906e\u65ad\u30fbModal \u306e\u6a5f\u80fd\u3082\u4f7f\u3048\u306a\u3044\u30fbVolume \u306f\u8aad\u307f\u53d6\u308a\u5c02\u7528\u3002ExLlamaV3 \u306f pip \u306e\u516c\u958b\n\u30d1\u30c3\u30b1\u30fc\u30b8\u3067\u3001\u914d\u5e03\u5143\u306e\u30ea\u30dd\u30b8\u30c8\u30ea\u306e\u30b3\u30fc\u30c9\u306f\u5165\u308c\u306a\u3044(\u914d\u5e03\u5143\u304c\u5225\u306b\u914d\u3063\u3066\u3044\u308b\u30ab\u30fc\u30cd\u30eb\u3082\u4f7f\u308f\u306a\u3044)\u3002\n\u8a18\u4e8b\u57f7\u7b46\u7528\u306e LLM \u306f\u547c\u3070\u306a\u3044\u3002\u56de\u7b54\u306e\u826f\u3057\u60aa\u3057\u3082\u5224\u5b9a\u3057\u306a\u3044(\u539f\u6587\u3092\u8fd4\u3059\u3060\u3051)\u3002\n&quot;&quot;&quot;\nimport json\nimport os\nimport platform\nimport time\nfrom datetime import datetime, timezone\n\nfrom lmw.lab.remote_code_remote import MOUNT, _log, installed_packages, plain, split_thinking\n\nfrom lmw.config import (\n    LAB_EXL3_CUDA_ARCHS, LAB_EXL3_CUDA_IMAGE, LAB_EXL3_PYTHON, LAB_EXL3_TORCH, LAB_EXL3_TORCH_INDEX, LAB_EXL3_VERSION,\n)\n\n# \u5b9f\u884c\u7528\u306e\u30a4\u30e1\u30fc\u30b8\u306e\u4f5c\u308a\u65b9(lmw\/lab\/gpu_modal.py \u304c\u5b9f\u884c\u3057\u3001\u8a18\u4e8b\u306e\u624b\u9806\u306b\u3082\u540c\u3058\u5024\u3092\u8f09\u305b\u308b)\nBASE_IMAGE = LAB_EXL3_CUDA_IMAGE\nPYTHON_VERSION = LAB_EXL3_PYTHON\nBUILD_ENV = {&quot;TORCH_CUDA_ARCH_LIST&quot;: LAB_EXL3_CUDA_ARCHS, &quot;MAX_JOBS&quot;: &quot;4&quot;}\nINSTALL_COMMANDS = (\n    f&quot;pip install torch=={LAB_EXL3_TORCH} --index-url {LAB_EXL3_TORCH_INDEX}&quot;,\n    &quot;pip install transformers jinja2 ninja setuptools wheel&quot;,\n    f&quot;pip install --no-build-isolation exllamav3=={LAB_EXL3_VERSION}&quot;,\n)\n\ndef _stop_ids(path: str, tokenizer) -&gt; list[int]:\n    &quot;&quot;&quot;generation_config.json \u306e eos(\u6574\u6570\u304b\u4e00\u89a7)\u3002\u7121\u3051\u308c\u3070\u30c8\u30fc\u30af\u30ca\u30a4\u30b6\u306e eos\u3002&quot;&quot;&quot;\n    ids: list[int] = []\n    try:\n        with open(os.path.join(path, &quot;generation_config.json&quot;), encoding=&quot;utf-8&quot;) as f:\n            eos = json.load(f).get(&quot;eos_token_id&quot;)\n        ids = [int(x) for x in (eos if isinstance(eos, list) else [eos])] if eos is not None else []\n    except (OSError, ValueError):\n        pass\n    if not ids and tokenizer.eos_token_id is not None:\n        ids = [int(tokenizer.eos_token_id)]\n    return ids\n\ndef _prompt_text(path: str, text: str) -&gt; str:\n    from transformers import AutoTokenizer\n    tok = AutoTokenizer.from_pretrained(path, local_files_only=True)\n    if not getattr(tok, &quot;chat_template&quot;, None):\n        return &quot;&quot;\n    messages = [{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: text}]\n    try:\n        return tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)\n    except TypeError:\n        return tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)\n\ndef _dequantize_embedding(src: str) -&gt; tuple[str, str]:\n    &quot;&quot;&quot;\n    \u57cb\u3081\u8fbc\u307f\u304c qweight(int8)+scales(\u884c\u3054\u3068\u306e bf16)\u3067\u4fdd\u5b58\u3055\u308c\u3066\u3044\u308b\u914d\u5e03(quantization_config.json \u306f\n    bf16 \u306e weight \u3068\u66f8\u3044\u3066\u3044\u308b)\u3092\u3001ExLlamaV3 \u304c\u8aad\u3081\u308b weight \u306b\u76f4\u3057\u305f\u4f5c\u696d\u7528\u30d5\u30a9\u30eb\u30c0\u3092 \/tmp \u306b\u4f5c\u308b\n    (Volume \u306f\u8aad\u307f\u53d6\u308a\u5c02\u7528)\u3002\u623b\u308a\u5024: (\u30d5\u30a9\u30eb\u30c0, \u76f4\u3057\u305f\u3053\u3068\u306e\u8aac\u660e)\u3002\u57cb\u3081\u8fbc\u307f\u304c\u6a19\u6e96\u306e\u5f62\u306a\u3089\u4f55\u3082\u3057\u306a\u3044\u3002\n    &quot;&quot;&quot;\n    import glob\n    import shutil\n    import torch\n    from safetensors import safe_open\n    from safetensors.torch import save_file\n    prefix = &quot;model.language_model.embed_tokens&quot;\n    holder = None\n    for f in sorted(glob.glob(os.path.join(src, &quot;*.safetensors&quot;))):\n        with safe_open(f, &quot;pt&quot;) as st:\n            keys = set(st.keys())\n        if f&quot;{prefix}.weight&quot; in keys:\n            return src, &quot;&quot;\n        if f&quot;{prefix}.qweight&quot; in keys:\n            holder = f\n    if holder is None:\n        return src, &quot;&quot;\n    with safe_open(holder, &quot;pt&quot;) as st:\n        q = st.get_tensor(f&quot;{prefix}.qweight&quot;)\n        scales = st.get_tensor(f&quot;{prefix}.scales&quot;)\n    weight = (q.to(torch.float32) * scales.to(torch.float32)[:, None]).to(torch.bfloat16).contiguous()\n    work = &quot;\/tmp\/exl3-model&quot;\n    shutil.rmtree(work, ignore_errors=True)\n    os.makedirs(work)\n    for name in os.listdir(src):\n        os.symlink(os.path.join(src, name), os.path.join(work, name))\n    save_file({f&quot;{prefix}.weight&quot;: weight}, os.path.join(work, &quot;embed-dequantized.safetensors&quot;))\n    return work, &quot;\u57cb\u3081\u8fbc\u307f: int8 \u306e qweight \u00d7 \u884c\u3054\u3068\u306e scales \u3092 bf16 \u306e weight \u306b\u76f4\u3057\u3066\u8aad\u307f\u8fbc\u307f&quot;\n\ndef run(job: dict) -&gt; dict:\n    &quot;&quot;&quot;\n    job: {&quot;dir&quot;, &quot;prompts&quot;: [{&quot;id&quot;, &quot;ja&quot;}], &quot;max_tokens&quot;, &quot;cache_tokens&quot;, &quot;answer_budget_sec&quot;}\u3002\n    \u30c1\u30e3\u30c3\u30c8\u30c6\u30f3\u30d7\u30ec\u30fc\u30c8\u304c\u7121\u3051\u308c\u3070 {&quot;no_chat_template&quot;: True}\u3002\n    &quot;&quot;&quot;\n    import exllamav3\n    import torch\n    from exllamav3 import Cache, Config, Generator, Job, Model, Tokenizer\n    from exllamav3.generator.sampler import CustomSampler, SS_Argmax\n    os.environ[&quot;HF_HUB_OFFLINE&quot;] = &quot;1&quot;\n    os.environ[&quot;TRANSFORMERS_OFFLINE&quot;] = &quot;1&quot;\n    t0 = time.time()\n    path = os.path.join(MOUNT, job[&quot;dir&quot;])\n    gpu = torch.cuda.get_device_name(0)\n    vram_total = torch.cuda.get_device_properties(0).total_memory\n    from importlib import metadata\n    version = getattr(exllamav3, &quot;__version__&quot;, &quot;&quot;) or metadata.version(&quot;exllamav3&quot;)\n    _log(t0, f&quot;GPU: {gpu}\u30fbexllamav3 {version}\u30fbtorch {torch.__version__}&quot;)\n    with open(os.path.join(path, &quot;quantization_config.json&quot;), encoding=&quot;utf-8&quot;) as f:\n        quant = json.load(f)\n    record = {&quot;runtime&quot;: &quot;exllamav3&quot;, &quot;gpu&quot;: gpu, &quot;vram_total&quot;: int(vram_total), &quot;exllamav3&quot;: str(version),\n              &quot;torch&quot;: str(torch.__version__), &quot;packages&quot;: installed_packages(),\n              &quot;python&quot;: platform.python_version(), &quot;cache_tokens&quot;: int(job[&quot;cache_tokens&quot;]),\n              &quot;quant_method&quot;: quant.get(&quot;quant_method&quot;), &quot;quant_version&quot;: quant.get(&quot;version&quot;),\n              &quot;bits&quot;: quant.get(&quot;bits&quot;), &quot;head_bits&quot;: quant.get(&quot;head_bits&quot;),\n              &quot;measured_at&quot;: datetime.now(timezone.utc).isoformat()}\n    first = _prompt_text(path, job[&quot;prompts&quot;][0][&quot;ja&quot;])\n    if not first:\n        _log(t0, &quot;\u30c1\u30e3\u30c3\u30c8\u30c6\u30f3\u30d7\u30ec\u30fc\u30c8\u304c\u3042\u308a\u307e\u305b\u3093&quot;)\n        return plain({**record, &quot;no_chat_template&quot;: True})\n    path, note = _dequantize_embedding(path)\n    if note:\n        record[&quot;embed_dequant&quot;] = note\n        _log(t0, note)\n    config = Config.from_directory(path)\n    model = Model.from_config(config)\n    cache = Cache(model, max_num_tokens=job[&quot;cache_tokens&quot;])\n    model.load(progressbar=False)\n    tokenizer = Tokenizer.from_config(config)\n    generator = Generator(model=model, cache=cache, tokenizer=tokenizer)\n    torch.cuda.synchronize()\n    record[&quot;vram_loaded&quot;] = int(torch.cuda.memory_allocated())\n    record[&quot;load_sec&quot;] = round(time.time() - t0)\n    _log(t0, f&quot;\u8aad\u307f\u8fbc\u307f: VRAM {record['vram_loaded'] \/ 1024 ** 3:.1f}GB\u30fb{record['load_sec']}\u79d2&quot;)\n    torch.cuda.reset_peak_memory_stats()\n    stops = _stop_ids(path, tokenizer)\n    sampler = CustomSampler([SS_Argmax()])\n    outputs = []\n    stop_at = t0 + job.get(&quot;answer_budget_sec&quot;, 10 ** 9)\n    for prompt in job[&quot;prompts&quot;]:\n        slowest = min((o[&quot;gen_tps&quot;] for o in outputs if o.get(&quot;gen_tps&quot;)), default=None)\n        worst = job[&quot;max_tokens&quot;] \/ slowest if slowest else 0\n        if time.time() + worst &gt; stop_at:\n            _log(t0, f&quot;\u6642\u9593\u306e\u4e0a\u9650\u304c\u8fd1\u3044\u305f\u3081\u3001\u6b8b\u308a\u306e\u554f\u3044({prompt['id']} \u304b\u3089)\u306f\u5c0b\u306d\u307e\u305b\u3093&quot;)\n            record[&quot;unanswered&quot;] = [p[&quot;id&quot;] for p in job[&quot;prompts&quot;][len(outputs):]]\n            break\n        text = _prompt_text(path, prompt[&quot;ja&quot;])\n        ids = tokenizer.encode(text, add_bos=False, encode_special_tokens=True)\n        started = time.time()\n        gen_job = Job(input_ids=ids, max_new_tokens=job[&quot;max_tokens&quot;], stop_conditions=stops, sampler=sampler,\n                      decode_special_tokens=True)\n        generator.enqueue(gen_job)\n        pieces: list[str] = []\n        count = 0\n        eos_reason = &quot;&quot;\n        while generator.num_remaining_jobs():\n            for result in generator.iterate():\n                if result.get(&quot;stage&quot;) != &quot;streaming&quot;:\n                    continue\n                pieces.append(result.get(&quot;text&quot;, &quot;&quot;))\n                if result.get(&quot;eos&quot;):\n                    count = int(result.get(&quot;new_tokens&quot;, 0))\n                    eos_reason = str(result.get(&quot;eos_reason&quot;, &quot;&quot;))\n        torch.cuda.synchronize()\n        seconds = time.time() - started\n        answer, thinking = split_thinking(&quot;&quot;.join(pieces), text[-64:])\n        for special in (&quot;&lt;|im_end|&gt;&quot;, &quot;&lt;|endoftext|&gt;&quot;, &quot;&lt;|im_start|&gt;&quot;):\n            answer = answer.replace(special, &quot;&quot;)\n        outputs.append({\n            &quot;id&quot;: prompt[&quot;id&quot;],\n            &quot;content&quot;: answer.strip(),\n            &quot;reasoning_chars&quot;: len(thinking.strip()),\n            &quot;finish_reason&quot;: &quot;length&quot; if eos_reason == &quot;max_new_tokens&quot; or count &gt;= job[&quot;max_tokens&quot;] else &quot;stop&quot;,\n            &quot;completion_tokens&quot;: count,\n            &quot;gen_tps&quot;: round(count \/ seconds, 2) if seconds &gt; 0 and count else None,\n            &quot;prompt_tokens&quot;: int(ids.shape[-1]),\n        })\n        _log(t0, f&quot;\u56de\u7b54 {prompt['id']}: {count}\u30c8\u30fc\u30af\u30f3\u30fb{outputs[-1]['gen_tps']} \u30c8\u30fc\u30af\u30f3\/\u79d2&quot;)\n    record[&quot;vram_peak&quot;] = int(torch.cuda.max_memory_allocated())\n    record[&quot;outputs&quot;] = outputs\n    gen = sorted(o[&quot;gen_tps&quot;] for o in outputs if o.get(&quot;gen_tps&quot;))\n    if gen:\n        record[&quot;tg_tps&quot;] = gen[len(gen) \/\/ 2]\n    return plain(record)\n<\/code><\/pre>\n<p>The <code>job<\/code> passed to <code>download<\/code> (as used):<\/p>\n<pre><code class=\"language-json\">{\n  &quot;repo&quot;: &quot;orcarouter\/OrcaSAQ-2-27B&quot;,\n  &quot;revision&quot;: &quot;15d20d7e9ae4fd89d1a47878f69381760169445b&quot;,\n  &quot;files&quot;: [\n    &quot;chat_template.jinja&quot;,\n    &quot;config.json&quot;,\n    &quot;generation_config.json&quot;,\n    &quot;merges.txt&quot;,\n    &quot;model-00001-of-00004.safetensors&quot;,\n    &quot;model-00002-of-00004.safetensors&quot;,\n    &quot;model-00003-of-00004.safetensors&quot;,\n    &quot;model-00004-of-00004.safetensors&quot;,\n    &quot;model.safetensors.index.json&quot;,\n    &quot;preprocessor_config.json&quot;,\n    &quot;quantization_config.json&quot;,\n    &quot;tokenizer.json&quot;,\n    &quot;tokenizer_config.json&quot;,\n    &quot;video_preprocessor_config.json&quot;,\n    &quot;vocab.json&quot;\n  ],\n  &quot;dir&quot;: &quot;run-1790831352&quot;\n}\n<\/code><\/pre>\n<p>The <code>job<\/code> passed to <code>run<\/code> (as used):<\/p>\n<pre><code class=\"language-json\">{\n  &quot;dir&quot;: &quot;run-1790831352&quot;,\n  &quot;max_tokens&quot;: 1024,\n  &quot;cache_tokens&quot;: 16384,\n  &quot;answer_budget_sec&quot;: 2220,\n  &quot;prompts&quot;: [\n    {\n      &quot;id&quot;: &quot;summary&quot;,\n      &quot;ja&quot;: &quot;\u6b21\u306e\u8a18\u4e8b\u30923\u6587\u3067\u8981\u7d04\u3057\u3066\u304f\u3060\u3055\u3044\u3002\\n\\n\u307f\u3069\u308a\u5e02\u306f\u4eca\u5e74\u306e\u6625\u3001\u5e02\u5185\u306b\u4e09\u3064\u3042\u308b\u516c\u7acb\u56f3\u66f8\u9928\u306e\u3046\u3061\u6700\u3082\u53e4\u3044\u4e2d\u592e\u56f3\u66f8\u9928\u306e\u6539\u4fee\u3092\u7d42\u3048\u3001\u5341\u6708\u4e00\u65e5\u306b\u518d\u958b\u9928\u3057\u305f\u3002\u6539\u4fee\u306f\u7d04\u4e00\u5e74\u534a\u306b\u308f\u305f\u308a\u3001\u7dcf\u4e8b\u696d\u8cbb\u306f\u7d04\u5341\u4e8c\u5104\u5186\u3002\u3053\u306e\u3046\u3061\u7d04\u56db\u5104\u5186\u306f\u56fd\u306e\u88dc\u52a9\u91d1\u3067\u307e\u304b\u306a\u308f\u308c\u305f\u3002\\n\\n\u6700\u5927\u306e\u5909\u5316\u306f\u3001\u95b2\u89a7\u5e2d\u306e\u914d\u7f6e\u3067\u3042\u308b\u3002\u4ee5\u524d\u306f\u9928\u5185\u306e\u5ea7\u5e2d\u306e\u5927\u534a\u304c\u5b66\u7fd2\u7528\u306e\u673a\u3067\u3001\u8a66\u9a13\u671f\u9593\u306b\u306a\u308b\u3068\u4e2d\u9ad8\u751f\u3067\u57cb\u307e\u308a\u3001\u9ad8\u9f62\u306e\u5229\u7528\u8005\u3084\u5c0f\u3055\u306a\u5b50\u3069\u3082\u9023\u308c\u304c\u5ea7\u308c\u306a\u3044\u3068\u3044\u3046\u82e6\u60c5\u304c\u6bce\u5e74\u5bc4\u305b\u3089\u308c\u3066\u3044\u305f\u3002\u6539\u4fee\u5f8c\u306f\u5ea7\u5e2d\u6570\u3092\u767e\u4e8c\u5341\u5e2d\u304b\u3089\u4e8c\u767e\u5e2d\u306b\u5897\u3084\u3057\u305f\u3046\u3048\u3067\u3001\u5b66\u7fd2\u7528\u306e\u673a\u3092\u4e09\u968e\u306b\u307e\u3068\u3081\u3001\u4e00\u968e\u306f\u4f1a\u8a71\u304c\u3067\u304d\u308b\u300c\u306b\u304e\u308f\u3044\u30d5\u30ed\u30a2\u300d\u3001\u4e8c\u968e\u306f\u9759\u304b\u306b\u672c\u3092\u8aad\u3080\u300c\u95b2\u89a7\u30d5\u30ed\u30a2\u300d\u306b\u5206\u3051\u305f\u3002\u4e00\u968e\u306b\u306f\u4e88\u7d04\u5236\u306e\u30b0\u30eb\u30fc\u30d7\u5b66\u7fd2\u5ba4\u304c\u56db\u5ba4\u3068\u3001\u98f2\u307f\u7269\u3092\u6301\u3061\u8fbc\u3081\u308b\u30ab\u30d5\u30a7\u30b9\u30da\u30fc\u30b9\u3082\u8a2d\u3051\u3089\u308c\u305f\u3002\\n\\n\u8535\u66f8\u306e\u9762\u3067\u3082\u898b\u76f4\u3057\u304c\u9032\u3093\u3060\u3002\u8cb8\u51fa\u306e\u5c11\u306a\u3044\u53e4\u3044\u8cc7\u6599\u7d04\u4e09\u4e07\u518a\u3092\u5e02\u5185\u306e\u4ed6\u306e\u9928\u3084\u4fdd\u5b58\u66f8\u5eab\u306b\u79fb\u3057\u3001\u4ee3\u308f\u308a\u306b\u5b50\u3069\u3082\u5411\u3051\u306e\u7d75\u672c\u3068\u3001\u4ecb\u8b77\u30fb\u5b50\u80b2\u3066\u30fb\u4ed5\u4e8b\u63a2\u3057\u3068\u3044\u3063\u305f\u66ae\u3089\u3057\u306b\u95a2\u308f\u308b\u5b9f\u7528\u66f8\u3092\u7d04\u4e00\u4e07\u4e94\u5343\u518a\u5897\u3084\u3057\u305f\u3002\u96fb\u5b50\u66f8\u7c4d\u306e\u8cb8\u51fa\u30b5\u30fc\u30d3\u30b9\u3082\u59cb\u307e\u308a\u3001\u5229\u7528\u767b\u9332\u3092\u3059\u308c\u3070\u81ea\u5b85\u304b\u3089\u4e00\u5ea6\u306b\u4e09\u518a\u307e\u3067\u3001\u4e8c\u9031\u9593\u501f\u308a\u3089\u308c\u308b\u3002\\n\\n\u5efa\u7269\u305d\u306e\u3082\u306e\u306b\u3082\u624b\u304c\u5165\u3063\u305f\u3002\u7bc9\u56db\u5341\u5e74\u3092\u8d85\u3048\u3066\u3044\u305f\u7a7a\u8abf\u8a2d\u5099\u3092\u5165\u308c\u66ff\u3048\u3001\u5c4b\u4e0a\u306b\u306f\u592a\u967d\u5149\u30d1\u30cd\u30eb\u3092\u8a2d\u7f6e\u3057\u3066\u3001\u9928\u5185\u3067\u4f7f\u3046\u96fb\u529b\u306e\u7d04\u4e8c\u5272\u3092\u307e\u304b\u306a\u3048\u308b\u3088\u3046\u306b\u3057\u305f\u3002\u8eca\u3044\u3059\u306e\u5229\u7528\u8005\u304b\u3089\u8981\u671b\u306e\u591a\u304b\u3063\u305f\u30a8\u30ec\u30d9\u30fc\u30bf\u30fc\u306f\u4e00\u57fa\u304b\u3089\u4e8c\u57fa\u306b\u5897\u3048\u3001\u66f8\u67b6\u306e\u9593\u306e\u901a\u8def\u3082\u4ee5\u524d\u3088\u308a\u4e09\u5341\u30bb\u30f3\u30c1\u5e83\u3052\u3089\u308c\u305f\u3002\u5165\u53e3\u306b\u306f\u6bb5\u5dee\u306e\u306a\u3044\u30b9\u30ed\u30fc\u30d7\u304c\u65b0\u305f\u306b\u8a2d\u3051\u3089\u308c\u3001\u6388\u4e73\u5ba4\u3068\u591a\u76ee\u7684\u30c8\u30a4\u30ec\u3082\u5404\u968e\u306b\u4e00\u3064\u305a\u3064\u7f6e\u304b\u308c\u305f\u3002\\n\\n\u4e00\u65b9\u3067\u8ab2\u984c\u3082\u6b8b\u308b\u3002\u99d0\u8eca\u5834\u306f\u6539\u4fee\u524d\u3068\u540c\u3058\u56db\u5341\u53f0\u306e\u307e\u307e\u3067\u3001\u9031\u672b\u306b\u306f\u8fd1\u304f\u306e\u9053\u8def\u306b\u9806\u756a\u5f85\u3061\u306e\u8eca\u304c\u4e26\u3093\u3060\u3002\u5e02\u306f\u96a3\u63a5\u3059\u308b\u5e02\u6709\u5730\u3092\u81e8\u6642\u99d0\u8eca\u5834\u3068\u3057\u3066\u958b\u653e\u3057\u305f\u304c\u3001\u6052\u4e45\u7684\u306a\u5bfe\u7b56\u306f\u6c7a\u307e\u3063\u3066\u3044\u306a\u3044\u3002\u307e\u305f\u3001\u958b\u9928\u6642\u9593\u3092\u5e73\u65e5\u306e\u591c\u4e5d\u6642\u307e\u3067\u5ef6\u3070\u3057\u305f\u3053\u3068\u3067\u8077\u54e1\u306e\u8ca0\u62c5\u304c\u5897\u3048\u3066\u304a\u308a\u3001\u8077\u54e1\u56e3\u4f53\u306f\u4eba\u54e1\u306e\u8ffd\u52a0\u3092\u6c42\u3081\u3066\u3044\u308b\u3002\\n\\n\u5229\u7528\u8005\u306e\u53d7\u3051\u6b62\u3081\u306f\u304a\u304a\u3080\u306d\u597d\u610f\u7684\u3060\u3002\u9031\u306b\u4e8c\u5ea6\u3001\u5b6b\u3092\u9023\u308c\u3066\u6765\u308b\u3068\u3044\u3046\u4e03\u5341\u4ee3\u306e\u5973\u6027\u306f\u300c\u524d\u306f\u5b50\u3069\u3082\u304c\u58f0\u3092\u51fa\u3059\u3068\u5468\u308a\u306e\u76ee\u304c\u6c17\u306b\u306a\u3063\u305f\u304c\u3001\u4e00\u968e\u306a\u3089\u6c17\u517c\u306d\u306a\u304f\u7d75\u672c\u3092\u8aad\u3093\u3067\u3042\u3052\u3089\u308c\u308b\u300d\u3068\u8a71\u3059\u3002\u4e00\u65b9\u3001\u53d7\u9a13\u3092\u63a7\u3048\u305f\u9ad8\u6821\u751f\u306e\u4e2d\u306b\u306f\u300c\u5b66\u7fd2\u5e2d\u304c\u4e09\u968e\u3060\u3051\u306b\u306a\u308a\u3001\u5915\u65b9\u306f\u3059\u3050\u306b\u6e80\u5e2d\u306b\u306a\u308b\u300d\u3068\u4e0d\u6e80\u3092\u6f0f\u3089\u3059\u4eba\u3082\u3044\u308b\u3002\u56f3\u66f8\u9928\u306f\u3001\u5b66\u7fd2\u5e2d\u306e\u7a7a\u304d\u72b6\u6cc1\u3092\u30db\u30fc\u30e0\u30da\u30fc\u30b8\u3067\u78ba\u8a8d\u3067\u304d\u308b\u4ed5\u7d44\u307f\u3092\u5e74\u5185\u306b\u5c0e\u5165\u3059\u308b\u4e88\u5b9a\u3060\u3002\\n\\n\u518d\u958b\u9928\u306b\u5408\u308f\u305b\u3066\u3001\u50ac\u3057\u3082\u5897\u3048\u305f\u3002\u6bce\u9031\u571f\u66dc\u306e\u5348\u524d\u306b\u306f\u8aad\u307f\u805e\u304b\u305b\u4f1a\u3001\u6708\u306b\u4e00\u5ea6\u306f\u5730\u5143\u306e\u8fb2\u5bb6\u3084\u5546\u5e97\u4e3b\u3092\u8b1b\u5e2b\u306b\u62db\u304f\u300c\u307e\u3061\u306e\u5148\u751f\u300d\u8b1b\u5ea7\u304c\u958b\u304b\u308c\u308b\u3002\u5341\u4e00\u6708\u306b\u306f\u3001\u5e02\u5185\u306e\u4e2d\u5b66\u751f\u304c\u9078\u3093\u3060\u672c\u3092\u7d39\u4ecb\u3059\u308b\u5c55\u793a\u3082\u4e88\u5b9a\u3055\u308c\u3066\u3044\u308b\u3002\\n\\n\u9928\u9577\u306e\u5c71\u672c\u548c\u5b50\u3055\u3093\u306f\u300c\u672c\u3092\u501f\u308a\u308b\u5834\u6240\u304b\u3089\u3001\u4eba\u304c\u96c6\u307e\u3063\u3066\u904e\u3054\u3059\u5834\u6240\u3078\u5909\u3048\u305f\u304b\u3063\u305f\u3002\u518d\u958b\u9928\u304b\u3089\u306e\u4e00\u304b\u6708\u3067\u6765\u9928\u8005\u306f\u4ee5\u524d\u306e\u4e00\u30fb\u516b\u500d\u306b\u306a\u3063\u305f\u300d\u3068\u8a71\u3059\u3002\u5e02\u306f\u6765\u5e74\u5ea6\u3001\u6b8b\u308b\u4e8c\u3064\u306e\u56f3\u66f8\u9928\u306b\u3064\u3044\u3066\u3082\u5229\u7528\u8005\u3078\u306e\u805e\u304d\u53d6\u308a\u3092\u884c\u3044\u3001\u6539\u4fee\u306e\u5fc5\u8981\u6027\u3092\u691c\u8a0e\u3059\u308b\u65b9\u91dd\u3060\u3002&quot;\n    },\n    {\n      &quot;id&quot;: &quot;keigo&quot;,\n      &quot;ja&quot;: &quot;\u53d6\u5f15\u5148\u306e\u62c5\u5f53\u8005\u306b\u3001\u6765\u9031\u706b\u66dc\u65e5\u306e\u6253\u3061\u5408\u308f\u305b\u3092\u6c34\u66dc\u65e5\u306b\u5909\u66f4\u3057\u3066\u307b\u3057\u3044\u3068\u304a\u9858\u3044\u3059\u308b\u30e1\u30fc\u30eb\u3092\u3001\u4e01\u5be7\u306a\u656c\u8a9e\u3067\u66f8\u3044\u3066\u304f\u3060\u3055\u3044\u3002&quot;\n    },\n    {\n      &quot;id&quot;: &quot;arithmetic&quot;,\n      &quot;ja&quot;: &quot;\u308a\u3093\u3054\u304c1\u500b120\u5186\u3001\u307f\u304b\u3093\u304c1\u500b45\u5186\u3067\u3059\u3002\u308a\u3093\u3054\u30923\u500b\u3068\u307f\u304b\u3093\u30928\u500b\u8cb7\u3063\u3066\u30011,000\u5186\u672d\u3067\u6255\u3044\u307e\u3057\u305f\u3002\u304a\u3064\u308a\u306f\u3044\u304f\u3089\u3067\u3059\u304b\u3002\u8003\u3048\u65b9\u3082\u793a\u3057\u3066\u304f\u3060\u3055\u3044\u3002&quot;\n    },\n    {\n      &quot;id&quot;: &quot;prefectures&quot;,\n      &quot;ja&quot;: &quot;\u65e5\u672c\u306e\u90fd\u9053\u5e9c\u770c\u306e\u3046\u3061\u3001\u6d77\u306b\u9762\u3057\u3066\u3044\u306a\u3044\u770c\u3092\u3059\u3079\u3066\u6319\u3052\u3066\u304f\u3060\u3055\u3044\u3002&quot;\n    },\n    {\n      &quot;id&quot;: &quot;haiku&quot;,\n      &quot;ja&quot;: &quot;\u300c\u6625\u300d\u3092\u30c6\u30fc\u30de\u306b\u3057\u305f\u4ff3\u53e5\u30923\u3064\u3001\u4e94\u30fb\u4e03\u30fb\u4e94\u306e\u97f3\u6570\u3092\u5b88\u3063\u3066\u4f5c\u3063\u3066\u304f\u3060\u3055\u3044\u3002\u305d\u308c\u305e\u308c\u306b\u4e00\u8a00\u306e\u8aac\u660e\u3092\u4ed8\u3051\u3066\u304f\u3060\u3055\u3044\u3002&quot;\n    }\n  ]\n}\n<\/code><\/pre>\n<details>\n<summary>Package versions in the running container (full list)<\/summary>\n<pre><code>aiohappyeyeballs==2.6.1 aiohttp==3.12.7 aiosignal==1.3.2 annotated-doc==0.0.5 annotated-types==0.8.0 anyio==4.15.1 attrs==25.3.0 cbor2==5.7.0 certifi==2026.7.22 click==8.5.0 exllamav3==1.5.3 filelock==3.32.3 frozenlist==1.6.0 fsspec==2026.7.0 grpclib==0.4.8 h11==0.16.0 h2==4.2.0 hf-xet==1.6.0 hpack==4.1.0 httpcore==1.0.9 httpx==0.28.1 huggingface_hub==1.33.0 hyperframe==6.1.0 idna==3.20 Jinja2==3.1.6 llguidance==1.9.0 marisa-trie==1.4.1 markdown-it-py==4.2.0 MarkupSafe==3.0.3 mdurl==0.1.2 mpmath==1.3.0 multidict==6.4.4 networkx==3.6.1 ninja==1.13.2 numpy==2.5.3 nvidia-cublas-cu12==12.8.4.1 nvidia-cuda-cupti-cu12==12.8.90 nvidia-cuda-nvrtc-cu12==12.8.93 nvidia-cuda-runtime-cu12==12.8.90 nvidia-cudnn-cu12==9.10.2.21 nvidia-cufft-cu12==11.3.3.83 nvidia-cufile-cu12==1.13.1.3 nvidia-curand-cu12==10.3.9.90 nvidia-cusolver-cu12==11.7.3.90 nvidia-cusparse-cu12==12.5.8.93 nvidia-cusparselt-cu12==0.7.1 nvidia-nccl-cu12==2.27.3 nvidia-nvjitlink-cu12==12.8.93 nvidia-nvtx-cu12==12.8.90 packaging==26.3 pillow==12.3.0 pip==23.3.2 propcache==0.3.1 protobuf==6.31.1 pydantic==2.13.5 pydantic_core==2.46.5 Pygments==2.21.0 PyYAML==6.0.3 regex==2026.9.29 rich==15.0.0 safetensors==0.8.0 setuptools==69.0.3 shellingham==1.5.4 sympy==1.14.0 tokenizers==0.23.2 torch==2.8.0+cu128 tqdm==4.70.1 transformers==5.17.0 triton==3.4.0 typer==0.27.2 typing-inspection==0.4.4 typing_extensions==4.16.0 wheel==0.48.0 yarl==1.20.0<\/code><\/pre>\n<\/details>\n<details>\n<summary>Package versions in the downloading container (full list)<\/summary>\n<pre><code>aiohappyeyeballs==2.6.1 aiohttp==3.12.7 aiosignal==1.3.2 anyio==4.15.1 attrs==25.3.0 cbor2==5.7.0 certifi==2025.4.26 click==8.5.0 filelock==4.0.7 frozenlist==1.6.0 fsspec==2026.9.0 grpclib==0.4.8 h11==0.16.0 h2==4.2.0 hf-xet==1.6.0 hpack==4.1.0 httpcore2==2.13.1 httpx2==2.13.1 huggingface_hub==2.0.0 hyperframe==6.1.0 idna==3.20 multidict==6.4.4 packaging==26.3 pip==25.1.1 propcache==0.3.1 protobuf==6.31.1 PyYAML==6.0.3 tqdm==4.70.1 truststore==0.10.4 typing_extensions==4.16.0 uv==0.7.19 wheel==0.45.1 yarl==1.20.0<\/code><\/pre>\n<\/details>\n<p><!-- \/lmw:lab --><\/p>\n<p><!-- lmw:peers --><\/p>\n<h2>Recent Models in the Same Size Class<\/h2>\n<p><em>Models with <\/em><em>15\u201340B<\/em><em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site&#8217;s estimates; licenses are as stated on the model cards.<\/em><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Model<\/th>\n<th>Parameters<\/th>\n<th>Smallest VRAM tier<\/th>\n<th>License<\/th>\n<th>Our article<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>elyza\/ELYZA-Thinking-1.0-llm-jp-4-32b-a3b<\/td>\n<td>32.1B<\/td>\n<td>80GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/02\/elyza-thinking-1-0-32b-33b-2\/\">ELYZA Releases ELYZA-Thinking-1.0 32B\/33B Reasoning Models<\/a> (2026-10-02)<\/td>\n<\/tr>\n<tr>\n<td>XingChen-AGI\/Xing4.0-29B-A4B<\/td>\n<td>31.2B<\/td>\n<td>24GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/28\/xing4-0-29b-a4b-china-telecom\/\">Xing4.0-29B-A4B Text Generation Model: Our Test Answers, 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n<tr>\n<td>Edge0\/Edge0-35B-A3B-preview<\/td>\n<td>36.0B<\/td>\n<td>24GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/11\/edge0-35b-a3b-preview-sparse-moe\/\">Edge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM<\/a> (2026-09-11)<\/td>\n<\/tr>\n<tr>\n<td>bartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n<td>26.5B<\/td>\n<td>12GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/11\/pantheon-reasoning-26b-gguf\/\">Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: Our Test Answers<\/a> (2026-09-11)<\/td>\n<\/tr>\n<tr>\n<td>nex-agi\/Nex-N2.5-mini<\/td>\n<td>35.1B<\/td>\n<td>16GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/09\/nex-n25-mini-released\/\">Nex-N2.5-mini Text Generation Model: Our Test Answers, 16GB+ VRAM<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><!-- \/lmw:peers --><\/p>\n<h2>How to Get It<\/h2>\n<ul>\n<li>Distribution: Hugging Face <code>orcarouter\/OrcaSAQ-2-27B<\/code> (safetensors, EXL3 format)<\/li>\n<li><strong>Does not run in llama.cpp, Ollama, or LM Studio.<\/strong> The execution kernel repository explicitly states: &#8220;Since llama.cpp can only read its own quantization codebooks, these weights cannot be used. Converting them to GGUF would require re-quantizing with a different quantizer&#8221;<\/li>\n<li><strong>When running with vLLM<\/strong>: In addition to vLLM, you need to install the publisher&#8217;s plugin (<code>Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>). The model card suggests <code>pip install git+https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/code>, and the kernel repository also provides startup configurations and Docker images tailored for 16GB. To correctly receive thinking outputs and tool calls, append <code>--reasoning-parser qwen3<\/code> and <code>--tool-call-parser qwen3_xml<\/code><\/li>\n<li><strong>When running with ExLlamaV3<\/strong>: The kernel repository notes that &#8220;for single-user use on consumer GPUs, it is lighter than vLLM and better suited.&#8221; However, you must first apply a patch to read embeddings packed in int8. Note that using MTP in ExLlamaV3 allocates separate caching for draft heads, increasing memory usage by about 10GB, which means it will not fit on a 16GB GPU<\/li>\n<li>Recommended sampling settings: temperature 1.0, top_p 0.95, top_k 20. Thinking mode is enabled by default<\/li>\n<li>The execution kernel repository was just created on September 24, making it very recent since release. It is wise to anticipate potential issues where it may not run smoothly depending on your environment<\/li>\n<li>License: Model is Apache-2.0 (inherited from the base Qwen3.8-27B). Please review the original license text before use<\/li>\n<\/ul>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/28\/hemmingway-1-open-27b-model-specialized-for-human-like-writing\/\">Hemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/12\/signal-3-8-27b-gguf-overview\/\">Signal-3.8-27B-GGUF: Our Test Answers, 16GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/26\/qwen3-8-27b-overview-specs-performance\/\">Qwen3.8-27B Vision-Language Model: Our Test Answers, 8GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/02\/cloudflare-clef-27b-multimodal-decision-model\/\">clef Structured Decision-Making Model: 12GB+ VRAM, GGUF Builds<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Find models by VRAM<\/strong> (This model runs from the 16GB tier) \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-16gb-en\/\">Other models that run on a 16GB GPU<\/a><\/li>\n<li><strong>Explore the same model family<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/model-qwen-qwen3-8-en\/\">Qwen3.8 family overview (8 articles, 17 converted builds)<\/a><\/li>\n<li><strong>What EXL3 mean and where to get this model<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-exl-en\/\">EXL2 \/ EXL3 format guide and models<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">Quantization and model-format glossary<\/a><\/li>\n<li><strong>Learn about the publisher<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/publisher-alibaba-en\/\">Alibaba (Qwen): models, licenses and articles<\/a><\/li>\n<li><strong>Other models for the same task<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/models-by-task-en\/#task-text\">Other text generation models<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B\">https:\/\/huggingface.co\/orcarouter\/OrcaSAQ-2-27B<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel\">https:\/\/github.com\/Continuum-AI-Corp\/OrcaSAQ2-kernel<\/a><\/li>\n<li><a href=\"https:\/\/huggingface.co\/Qwen\/Qwen3.8-27B\">https:\/\/huggingface.co\/Qwen\/Qwen3.8-27B<\/a><\/li>\n<\/ul>\n<p><!-- lmw:updates --><\/p>\n<h2>Update History<\/h2>\n<ul>\n<li>2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<li>2026-09-29: Verified the content against the official primary source.<\/li>\n<li>2026-10-01: Added our own measurements: VRAM and answers to Japanese questions with the publisher&#8217;s own code.<\/li>\n<li>2026-10-01: Changed the title to show what the article covers (VRAM requirements, file list, etc.).<\/li>\n<\/ul>\n<p><!-- \/lmw:updates --><\/p>\n","protected":false},"excerpt":{"rendered":"<p>OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 16GB GPU.<\/p>\n","protected":false},"author":1,"featured_media":8622,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[2592,2594,2596,522,1082,1547,592,1555],"class_list":["post-6423","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-exl3-en","tag-orcarouter-en","tag-orcarouter-orcasaq-2-27b-en","tag-qwen3-8-en","tag-qwen3-8-27b-en","tag-verified","tag-vllm-en","tag--en"],"lang":"en","translations":{"en":6423,"ja":6421},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6423"}],"version-history":[{"count":24,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions"}],"predecessor-version":[{"id":15783,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/6423\/revisions\/15783"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/8622"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=6423"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=6423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}