{"id":11858,"date":"2026-11-17T11:00:36","date_gmt":"2026-11-17T02:00:36","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/?p=11858"},"modified":"2026-11-17T11:09:47","modified_gmt":"2026-11-17T02:09:47","slug":"ifm-jais-3-37b-gguf","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/ifm-jais-3-37b-gguf\/","title":{"rendered":"Jais-3-3.7B-GGUF 512K Context Language Model: 4GB+ VRAM"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/huggingface.co\/IFM\/Jais-3-3.7B-GGUF\">IFM\/Jais-3-3.7B-GGUF<\/a><\/td>\n<\/tr>\n<tr>\n<td>Family guide<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/model-inception42-jais-3-en\/\">Jais-3 guide (4 articles)<\/a><\/td>\n<\/tr>\n<tr>\n<td>Publisher guide<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/publisher-ifm-en\/\">IFM (MBZUAI): models and licenses<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-10-13<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>apache-2.0<\/td>\n<\/tr>\n<tr>\n<td>Formats<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-gguf-en\/\">GGUF<\/a><\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>IFM has released &#8220;IFM\/Jais-3-3.7B-GGUF&#8221;, a GGUF-converted version of the open-weight model &#8220;inception42\/Jais-3-3.7B&#8221; which supports Arabic and English.<\/p>\n<p>Despite being a small-scale 3.7B class dense language model, this model features an exceptionally long context window of 512K tokens. It is designed to run on the inference framework &#8220;llama.cpp&#8221; and is provided for local inference processing and agent use cases involving tool calls.<\/p>\n<h2>Specifications<\/h2>\n<ul>\n<li>Number of parameters: 3.7B<\/li>\n<li>Number of activated parameters: 3.7B<\/li>\n<li>Architecture: Dense (K2HorizonForCausalLM \/ decoder-only)<\/li>\n<li>Context length: 512K<\/li>\n<\/ul>\n<h2>Performance<\/h2>\n<p>The table below shows benchmark scores published by the publisher IFM on the model card. It should be noted that this model&#8217;s measurements are based on the BF16 precision original model, and individual evaluations of the quantized models themselves have not yet been conducted. Comparable open-weight models of a similar scale are included as reference columns as-is.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th><\/th>\n<th>Jais-3-3.7B<\/th>\n<th>Reference models \/ Qwen3.5-4B<\/th>\n<th>Reference models \/ Gemma 4 E2B<\/th>\n<th>Reference models \/ granite-4.0-h-3b-ar<\/th>\n<th>Reference models \/ Falcon-H1-3B-Instruct<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td># Params<\/td>\n<td>3.7B<\/td>\n<td>4B<\/td>\n<td>5B<\/td>\n<td>3B<\/td>\n<td>3B<\/td>\n<\/tr>\n<tr>\n<td># Activated params<\/td>\n<td>3.7B<\/td>\n<td>4B<\/td>\n<td>2.3B<\/td>\n<td>3B<\/td>\n<td>3B<\/td>\n<\/tr>\n<tr>\n<td>Architecture<\/td>\n<td>Dense<\/td>\n<td>Dense<\/td>\n<td>Dense<\/td>\n<td>Dense<\/td>\n<td>Dense<\/td>\n<\/tr>\n<tr>\n<td>Open-ended Generation<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>AraGen<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>AraGen v3 (3C3H) 09-2025<\/td>\n<td>49.6<\/td>\n<td>45.8<\/td>\n<td>39.6<\/td>\n<td>27.1<\/td>\n<td>13.0<\/td>\n<\/tr>\n<tr>\n<td>Correctness<\/td>\n<td>57.9<\/td>\n<td>57.2<\/td>\n<td>46.5<\/td>\n<td>30.8<\/td>\n<td>16.8<\/td>\n<\/tr>\n<tr>\n<td>Completeness<\/td>\n<td>56.9<\/td>\n<td>56.3<\/td>\n<td>46.4<\/td>\n<td>30.4<\/td>\n<td>15.3<\/td>\n<\/tr>\n<tr>\n<td>Conciseness<\/td>\n<td>18.2<\/td>\n<td>10.3<\/td>\n<td>11.5<\/td>\n<td>14.4<\/td>\n<td>3.5<\/td>\n<\/tr>\n<tr>\n<td>Helpfulness<\/td>\n<td>51.9<\/td>\n<td>45.5<\/td>\n<td>41.5<\/td>\n<td>27.6<\/td>\n<td>13.4<\/td>\n<\/tr>\n<tr>\n<td>Honesty<\/td>\n<td>55.6<\/td>\n<td>49.0<\/td>\n<td>45.2<\/td>\n<td>28.9<\/td>\n<td>13.8<\/td>\n<\/tr>\n<tr>\n<td>Harmlessness<\/td>\n<td>56.9<\/td>\n<td>56.3<\/td>\n<td>46.5<\/td>\n<td>30.6<\/td>\n<td>15.4<\/td>\n<\/tr>\n<tr>\n<td>Knowledge &amp; Reasoning<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>General Knowledge &amp; Comprehension<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Arabic MMLU* Accuracy<\/td>\n<td>83.3<\/td>\n<td>75.2<\/td>\n<td>69.4<\/td>\n<td>59.8<\/td>\n<td>51.0<\/td>\n<\/tr>\n<tr>\n<td>Belebele Accuracy<\/td>\n<td>91.9<\/td>\n<td>84.0<\/td>\n<td>79.6<\/td>\n<td>61.9<\/td>\n<td>57.9<\/td>\n<\/tr>\n<tr>\n<td>STEM &amp; Math<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>AraSTEM* Accuracy<\/td>\n<td>91.7<\/td>\n<td>91.4<\/td>\n<td>85.1<\/td>\n<td>66.5<\/td>\n<td>64.9<\/td>\n<\/tr>\n<tr>\n<td>Arabic GSM8K* Accuracy<\/td>\n<td>91.9<\/td>\n<td>88.7<\/td>\n<td>81.7<\/td>\n<td>67.2<\/td>\n<td>72.2<\/td>\n<\/tr>\n<tr>\n<td>GATmath* Accuracy<\/td>\n<td>96.2<\/td>\n<td>95.9<\/td>\n<td>90.5<\/td>\n<td>45.3<\/td>\n<td>41.3<\/td>\n<\/tr>\n<tr>\n<td>Professional Domains<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>MedAraBench* Accuracy<\/td>\n<td>74.1<\/td>\n<td>74.1<\/td>\n<td>57.2<\/td>\n<td>38.1<\/td>\n<td>34.5<\/td>\n<\/tr>\n<tr>\n<td>Arabic Legal (HELM)* Accuracy<\/td>\n<td>66.7<\/td>\n<td>63.3<\/td>\n<td>57.1<\/td>\n<td>56.1<\/td>\n<td>53.0<\/td>\n<\/tr>\n<tr>\n<td>Arabic Language &amp; Culture<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Islamic Knowledge &amp; Arab Culture<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>IslamicQA Accuracy (mean of the 4 below)<\/td>\n<td>81.7<\/td>\n<td>61.6<\/td>\n<td>63.0<\/td>\n<td>41.2<\/td>\n<td>48.2<\/td>\n<\/tr>\n<tr>\n<td>PaLM-X 2025-2* Accuracy<\/td>\n<td>88.9<\/td>\n<td>79.6<\/td>\n<td>79.2<\/td>\n<td>57.5<\/td>\n<td>66.6<\/td>\n<\/tr>\n<tr>\n<td>QIAS-1* Accuracy<\/td>\n<td>83.3<\/td>\n<td>50.7<\/td>\n<td>51.6<\/td>\n<td>18.2<\/td>\n<td>27.5<\/td>\n<\/tr>\n<tr>\n<td>QIAS-2* Accuracy<\/td>\n<td>80.0<\/td>\n<td>69.9<\/td>\n<td>65.3<\/td>\n<td>66.5<\/td>\n<td>54.3<\/td>\n<\/tr>\n<tr>\n<td>IslamicEval2025-1B* Accuracy<\/td>\n<td>74.5<\/td>\n<td>46.3<\/td>\n<td>55.8<\/td>\n<td>22.5<\/td>\n<td>44.6<\/td>\n<\/tr>\n<tr>\n<td>Arab Culture Accuracy (mean of the 3 below)<\/td>\n<td>81.2<\/td>\n<td>68.8<\/td>\n<td>66.8<\/td>\n<td>60.2<\/td>\n<td>54.8<\/td>\n<\/tr>\n<tr>\n<td>ArabCulture* Accuracy<\/td>\n<td>84.3<\/td>\n<td>76.2<\/td>\n<td>74.7<\/td>\n<td>68.4<\/td>\n<td>64.2<\/td>\n<\/tr>\n<tr>\n<td>AraDiCE-Culture* Accuracy<\/td>\n<td>76.5<\/td>\n<td>61.5<\/td>\n<td>56.4<\/td>\n<td>53.6<\/td>\n<td>45.8<\/td>\n<\/tr>\n<tr>\n<td>Arabic Cuisine^ Accuracy<\/td>\n<td>82.6<\/td>\n<td>68.9<\/td>\n<td>69.2<\/td>\n<td>58.6<\/td>\n<td>54.4<\/td>\n<\/tr>\n<tr>\n<td>Dialects<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Dialectal Arabic MMLU* Accuracy<\/td>\n<td>81.6<\/td>\n<td>76.5<\/td>\n<td>65.1<\/td>\n<td>50.7<\/td>\n<td>43.8<\/td>\n<\/tr>\n<tr>\n<td>Dialect Identification (QADI) Accuracy<\/td>\n<td>68.8<\/td>\n<td>37.4<\/td>\n<td>44.4<\/td>\n<td>42.2<\/td>\n<td>28.5<\/td>\n<\/tr>\n<tr>\n<td>Alyah* Accuracy<\/td>\n<td>79.7<\/td>\n<td>63.3<\/td>\n<td>69.7<\/td>\n<td>66.0<\/td>\n<td>58.9<\/td>\n<\/tr>\n<tr>\n<td>Linguistics &amp; Literature<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Arabic MMLU with diacritics Accuracy<\/td>\n<td>80.7<\/td>\n<td>70.8<\/td>\n<td>64.7<\/td>\n<td>53.3<\/td>\n<td>34.6<\/td>\n<\/tr>\n<tr>\n<td>FannOrFlop-MCQ Accuracy<\/td>\n<td>66.2<\/td>\n<td>50.9<\/td>\n<td>60.0<\/td>\n<td>48.8<\/td>\n<td>41.1<\/td>\n<\/tr>\n<tr>\n<td>Arabic Linguistics^ Accuracy (mean of the 2 below)<\/td>\n<td>92.1<\/td>\n<td>72.7<\/td>\n<td>77.9<\/td>\n<td>49.3<\/td>\n<td>42.1<\/td>\n<\/tr>\n<tr>\n<td>Diacritization Accuracy<\/td>\n<td>86.9<\/td>\n<td>79.2<\/td>\n<td>74.2<\/td>\n<td>64.1<\/td>\n<td>51.2<\/td>\n<\/tr>\n<tr>\n<td>Parsing Accuracy<\/td>\n<td>97.3<\/td>\n<td>66.3<\/td>\n<td>81.6<\/td>\n<td>34.6<\/td>\n<td>32.9<\/td>\n<\/tr>\n<tr>\n<td>Assistant Capabilities<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Instruction Following<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Arabic IFBench^ Accuracy (prompt-level strict)<\/td>\n<td>92.6<\/td>\n<td>53.7<\/td>\n<td>37.6<\/td>\n<td>14.7<\/td>\n<td>18.4<\/td>\n<\/tr>\n<tr>\n<td>Arabic IFEval Accuracy<\/td>\n<td>79.0<\/td>\n<td>73.0<\/td>\n<td>69.6<\/td>\n<td>43.1<\/td>\n<td>55.7<\/td>\n<\/tr>\n<tr>\n<td>Factuality &amp; Hallucination<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>HalluScore Accuracy<\/td>\n<td>48.3<\/td>\n<td>37.5<\/td>\n<td>33.3<\/td>\n<td>16.4<\/td>\n<td>13.2<\/td>\n<\/tr>\n<tr>\n<td>SimpleQA Verified (Ar) Aggregate (2\/3 F-score + 1\/3 non-hallucination rate)<\/td>\n<td>29.1<\/td>\n<td>7.0<\/td>\n<td>29.8<\/td>\n<td>3.5<\/td>\n<td>2.5<\/td>\n<\/tr>\n<tr>\n<td>F-score<\/td>\n<td>1.5<\/td>\n<td>7.4<\/td>\n<td>2.2<\/td>\n<td>4.5<\/td>\n<td>2.3<\/td>\n<\/tr>\n<tr>\n<td>Non-hallucination rate<\/td>\n<td>84.2<\/td>\n<td>6.3<\/td>\n<td>84.8<\/td>\n<td>1.4<\/td>\n<td>3.0<\/td>\n<\/tr>\n<tr>\n<td>Agentic<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Terminal Bench 2.1 Accuracy (Avg@3)<\/td>\n<td>41.2<\/td>\n<td>25.8<\/td>\n<td>0.4<\/td>\n<td>0.0<\/td>\n<td>0.0<\/td>\n<\/tr>\n<tr>\n<td>GDPVal Accuracy (@ 75% threshold)<\/td>\n<td>54.5<\/td>\n<td>15.0<\/td>\n<td>3.2<\/td>\n<td>0.0<\/td>\n<td>0.0<\/td>\n<\/tr>\n<tr>\n<td>MCPMark Accuracy (Pass@1)<\/td>\n<td>30.7<\/td>\n<td>25.2<\/td>\n<td>1.6<\/td>\n<td>0.0<\/td>\n<td>0.0<\/td>\n<\/tr>\n<tr>\n<td>WildClawBench Mean Score@3<\/td>\n<td>40.9<\/td>\n<td>33.3<\/td>\n<td>18.2<\/td>\n<td>8.2<\/td>\n<td>0.1<\/td>\n<\/tr>\n<tr>\n<td>Apex-Agents Accuracy (Pass@1)<\/td>\n<td>16.7<\/td>\n<td>6.4<\/td>\n<td>0.0<\/td>\n<td>0.5<\/td>\n<td>0.2<\/td>\n<\/tr>\n<tr>\n<td>GAIA v2 lit AR Accuracy (Pass@1)<\/td>\n<td>45.5<\/td>\n<td>27.9<\/td>\n<td>18.2<\/td>\n<td>0.0<\/td>\n<td>0.0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>Looking at the published measurement results, the model outperforms comparison models in Arabic MMLU (83.3% accuracy), which measures general knowledge across 57 categories of 4-choice questions, and Arabic GSM8K (91.9% accuracy), which solves elementary school-level math problems. It also records high figures in agent-related and instruction-following metrics, such as Arabic IFEval (79.0%) measuring formal instruction compliance, Terminal Bench 2.1 (41.2%) measuring the ability to execute commands and perform tasks in a terminal, and GAIA v2 lit AR (45.5%) measuring the ability to solve research tasks.<\/p>\n<p>On the other hand, in SimpleQA Verified (Ar), which checks for fabricated information in short fact-based questions with a single correct answer, while the non-hallucination rate is high at 84.2%, the F-score remains at 1.5%, falling below Qwen3.5-4B&#8217;s 7.4% and granite-4.0-h-3b-ar&#8217;s 4.5%. While it shows strengths in Arabic-world knowledge and agent tasks overall, it reveals that a balance between precision and recall in specific short fact-checking tasks remains a challenge.<\/p>\n<h2>Strengths and Use Cases<\/h2>\n<p>Jais-3-3.7B is developed for conversational assistants targeting Arabic and English, as well as agent use cases that autonomously call external tools to perform tasks. Despite being a small-scale 3.7B class model, it features an ultra-long context length of up to 512K tokens, making it suitable for reading long texts and conducting dialogues that maintain complex contexts.<\/p>\n<p>According to the model card, specific areas of strength and operational tips for this model include the following:<\/p>\n<ul>\n<li><strong>Separation of Advanced Reasoning and Thought Processes<\/strong>: It features a thinking mode that performs logical progression inside the model, allowing API responses to output the thought content (<code>reasoning_content<\/code>) and final answer (<code>content<\/code>) separately. The publisher recommends specifying <code>reasoning_effort=\"high\"<\/code> as a per-request parameter to fully unleash its reasoning performance.<\/li>\n<li><strong>Tool Calling and Agent Capabilities<\/strong>: It has the capability to autonomously perform function calling. In addition to the default <code>xml<\/code>, the output format for tool calls also supports <code>json<\/code> and <code>xml_typed<\/code>, which can be switched via settings.<\/li>\n<li><strong>Language-Specific Recommended Generation Parameters<\/strong>: For sampling settings when using the thinking mode, <code>temperature=1.0<\/code> is recommended for English and <code>temperature=0.7<\/code> for Arabic (with <code>temperature=0.2<\/code> for default use, and a common <code>top_p=0.95<\/code> and <code>top_k=20<\/code>).<\/li>\n<\/ul>\n<p>Meanwhile, it is explicitly stated that usage in languages other than Arabic and English is out of scope. Furthermore, because it may contain outputs with content differing from facts (hallucinations) or biases derived from training data, it is considered unsuitable for critical judgment applications such as medical, legal, or financial use without human supervision.<\/p>\n<p><!-- lmw:hardware --><\/p>\n<h2>Hardware Requirements<\/h2>\n<p><strong>Estimated requirements (calculated by Local Model Watch)<\/strong> \u2014 5.1B parameters (taken from the base model inception42\/Jais-3-3.7B)<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Your VRAM<\/th>\n<th>Quantization<\/th>\n<th>File size<\/th>\n<th>Est. memory needed<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>4GB (laptop iGPU \/ phone class)<\/td>\n<td>Q5_0<\/td>\n<td>3.3GB<\/td>\n<td>4.0GB<\/td>\n<\/tr>\n<tr>\n<td>8GB (RTX 4060 \/ 3060 Ti, etc.)<\/td>\n<td>Q8_0<\/td>\n<td>5.0GB<\/td>\n<td>6.0GB<\/td>\n<\/tr>\n<tr>\n<td>12GB (RTX 4070 \/ 3060 12GB, etc.)<\/td>\n<td>BF16<\/td>\n<td>9.4GB<\/td>\n<td>11.3GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model&#8217;s authors. Compare with other models in our <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-en\/\">VRAM quick reference<\/a>. What the quantization names mean: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">glossary<\/a>.<\/em><\/p>\n<p><!-- \/lmw:hardware --><\/p>\n<p><!-- lmw:runnability --><\/p>\n<h2>Can You Run It Locally?<\/h2>\n<p><strong>Runs in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\n<p>It is distributed in GGUF, so no conversion is needed.<\/p>\n<p><strong>License \u2014 <code>apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\n<p><strong>Compression:<\/strong> the Q5_0 build measures 5.6 bits per weight \u2014 about 35% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\n<p><em>Compiled by this site&#8217;s code from the published formats, converted builds we have found, and each engine&#8217;s own model registry. &#8220;Not found&#8221; means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher&#8217;s original terms before relying on them.<\/em><\/p>\n<p><!-- \/lmw:runnability --><\/p>\n<p><!-- lmw:peers --><\/p>\n<h2>Recent Models in the Same Size Class<\/h2>\n<p><em>Models with <\/em><em>4\u201315B<\/em><em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site&#8217;s estimates; licenses are as stated on the model cards.<\/em><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Model<\/th>\n<th>Parameters<\/th>\n<th>Smallest VRAM tier<\/th>\n<th>License<\/th>\n<th>Our article<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>microsoft\/UserRepGen-4B<\/td>\n<td>4.5B<\/td>\n<td>4GB<\/td>\n<td>mit<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/userrepgen-4b\/\">UserRepGen-4B Text Generation Model: 4GB+ VRAM, GGUF Builds<\/a> (2026-11-17)<\/td>\n<\/tr>\n<tr>\n<td>IFM\/Jais-3-7B-GGUF<\/td>\n<td>9.0B<\/td>\n<td>8GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/jais-3-7b-gguf-local-inference\/\">Jais-3-7B-GGUF Arabic and English 7B Reasoning Model: 8GB+ VRAM<\/a> (2026-11-17)<\/td>\n<\/tr>\n<tr>\n<td>ibm-granite\/granite-switch-4.1-8b<\/td>\n<td>8.8B<\/td>\n<td>24GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/ibm-granite-switch-4-1-2\/\">IBM Releases Granite Switch 4.1: Modular LLMs with aLoRA and Speech<\/a> (2026-11-17)<\/td>\n<\/tr>\n<tr>\n<td>ggml-org\/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF<\/td>\n<td>8.0B<\/td>\n<td>12GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/nvidia-nemotron-3-5-bolt-e4b-gguf-2\/\">NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF Text Generation Model: 12GB+ VRAM<\/a> (2026-11-17)<\/td>\n<\/tr>\n<tr>\n<td>ByteDance-Seed\/Cola2<\/td>\n<td>8.8B<\/td>\n<td>48GB<\/td>\n<td>\u2014<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/bytedance-seed-releases-cola2-language-model\/\">ByteDance-Seed Releases Cola2 Language Model<\/a> (2026-11-17)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><!-- \/lmw:peers --><\/p>\n<h2>How to Get It<\/h2>\n<p>Because the model repository is not gated, it can be downloaded without prior application via a Hugging Face account or agreement to terms of use. The license is Apache 2.0.<\/p>\n<h3>Distribution Formats<\/h3>\n<p>In this repository (<code>IFM\/Jais-3-3.7B-GGUF<\/code>), files quantized in GGUF format for llama.cpp are distributed. The available variants are as follows:<\/p>\n<ul>\n<li><code>BF16<\/code><\/li>\n<li><code>Q8_0<\/code><\/li>\n<li><code>Q6_K<\/code><\/li>\n<li><code>Q5_K_M<\/code><\/li>\n<li><code>Q5_0<\/code><\/li>\n<li><code>Q4_K_M<\/code><\/li>\n<\/ul>\n<p>Additionally, the collection separately provides FP8 formats and MLX formats (4bit, 8bit, bf16) tailored for Apple Silicon environments.<\/p>\n<h3>Inference Engine and Startup Method<\/h3>\n<p>Support for the K2 Horizon architecture adopted by this model has been merged into a llama.cpp pull request (<code>ggml-org\/llama.cpp#29535<\/code>). Consequently, running it requires using a recent llama.cpp build (such as the master branch) that supports K2 Horizon.<\/p>\n<p>An example command to build in a CUDA environment and start <code>llama-server<\/code> is as follows:<\/p>\n<pre><code class=\"language-shell\">cmake -B build -DGGML_CUDA=ON\ncmake --build build -j --target llama-server\n.\/build\/bin\/llama-server -hf IFM\/Jais-3-3.7B-GGUF:Q8_0 --jinja -ngl 99 -c 32768\n<\/code><\/pre>\n<p>The startup option <code>-hf<\/code> allows directly fetching and executing variants such as <code>:Q8_0<\/code> or <code>:Q4_K_M<\/code> from the choices above. Additionally, to correctly process the thought process and tool-calling templates, it is recommended to pass the <code>--jinja<\/code> option when launching the server.<\/p>\n<h3>Example Usage via API<\/h3>\n<p>Here is an implementation example accessing the launched server from an OpenAI-compatible Python client. You can execute inference including the thought process.<\/p>\n<pre><code class=\"language-python\">from openai import OpenAI\n\nclient = OpenAI(base_url=&quot;http:\/\/localhost:8080\/v1&quot;, api_key=&quot;EMPTY&quot;)\nresponse = client.chat.completions.create(\n    model=&quot;Jais-3-3.7B&quot;,\n    messages=[\n        {\n            &quot;role&quot;: &quot;user&quot;,\n            &quot;content&quot;: &quot;\u0644\u062f\u064a\u0646\u0627 \u0639\u062f\u062f\u0627\u0646 \u0645\u062c\u0645\u0648\u0639\u0647\u0645\u0627 50 \u0648\u062d\u0627\u0635\u0644 \u0636\u0631\u0628\u0647\u0645\u0627 600. \u0645\u0627 \u0647\u0645\u0627 \u0627\u0644\u0639\u062f\u062f\u0627\u0646\u061f \u0627\u0634\u0631\u062d \u062e\u0637\u0648\u0627\u062a \u0627\u0644\u062d\u0644\u3002&quot;,\n        }\n    ],\n    temperature=0.7,\n    top_p=0.95,\n    max_tokens=32768,\n    extra_body={\n        &quot;top_k&quot;: 20,\n        &quot;chat_template_kwargs&quot;: {&quot;reasoning_effort&quot;: &quot;high&quot;},\n    },\n)\n\nmessage = response.choices[0].message\nprint(&quot;Reasoning:&quot;, getattr(message, &quot;reasoning_content&quot;, None))\nprint(&quot;Answer:&quot;, message.content)\n<\/code><\/pre>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/jais-3-0-9b-gguf-released\/\">Jais-3-0.9B-GGUF Bilingual Dense Model: 4GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/jais-3-7b-gguf-local-inference\/\">Jais-3-7B-GGUF Arabic and English 7B Reasoning Model: 8GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/jais-3-32b-gguf-released\/\">Jais-3-32B-GGUF Arabic and English 32B Dense Model: 24GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/jais-3-mova-36b-a4b-gguf\/\">Jais-3-MoVA-36B-A4B-GGUF Text Generation Model: 32GB+ VRAM<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Find models by VRAM<\/strong> (This model runs from the 4GB tier) \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-8gb-en\/\">Other models that run on a 8GB GPU<\/a><\/li>\n<li><strong>Explore the same model family<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/model-inception42-jais-3-en\/\">Jais-3 family overview (4 articles)<\/a><\/li>\n<li><strong>Engines that run this model<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-llama-cpp-en\/\">llama.cpp<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-ollama-en\/\">Ollama<\/a><\/li>\n<li><strong>What Q5_0, Q8_0 mean and where to get this model<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-gguf-en\/\">GGUF format guide and models<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">Quantization and model-format glossary<\/a><\/li>\n<li><strong>Learn about the publisher<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/publisher-ifm-en\/\">IFM (MBZUAI): models, licenses and articles<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/huggingface.co\/IFM\/Jais-3-3.7B-GGUF\">IFM\/Jais-3-3.7B-GGUF (Hugging Face)<\/a><\/li>\n<li><a href=\"https:\/\/huggingface.co\/inception42\/Jais-3-3.7B\">inception42\/Jais-3-3.7B (Hugging Face)<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>IFM has released IFM\/Jais-3-3.7B-GGUF, a GGUF version of the Arabic-English open-weight model supporting a 512K context window.<\/p>\n","protected":false},"author":1,"featured_media":11857,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[3428,312,163,2051,3417,3447,1073,1547,1555],"class_list":["post-11858","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-arabic-en","tag-dense-en","tag-gguf-en","tag-ifm-en","tag-jais-3-en","tag-jais-3-3-7b-en","tag-llama-cpp-en","tag-verified","tag--en"],"lang":"en","translations":{"en":11858,"ja":11856},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/11858","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=11858"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/11858\/revisions"}],"predecessor-version":[{"id":17799,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/11858\/revisions\/17799"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/11857"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=11858"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=11858"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=11858"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}