Jais-3-3.7B-GGUF 512K Context Language Model: 4GB+ VRAM

Jais-3-3.7B-GGUF 512K Context Language Model: 4GB+ VRAM

At a Glance

Item Value
Repository IFM/Jais-3-3.7B-GGUF
Family guide Jais-3 guide (4 articles)
Publisher guide IFM (MBZUAI): models and licenses
Published 2026-10-13
License apache-2.0
Formats GGUF
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

IFM has released “IFM/Jais-3-3.7B-GGUF", a GGUF-converted version of the open-weight model “inception42/Jais-3-3.7B" which supports Arabic and English.

Despite being a small-scale 3.7B class dense language model, this model features an exceptionally long context window of 512K tokens. It is designed to run on the inference framework “llama.cpp" and is provided for local inference processing and agent use cases involving tool calls.

Specifications

  • Number of parameters: 3.7B
  • Number of activated parameters: 3.7B
  • Architecture: Dense (K2HorizonForCausalLM / decoder-only)
  • Context length: 512K

Performance

The table below shows benchmark scores published by the publisher IFM on the model card. It should be noted that this model’s measurements are based on the BF16 precision original model, and individual evaluations of the quantized models themselves have not yet been conducted. Comparable open-weight models of a similar scale are included as reference columns as-is.

→ Scroll horizontally to see all columns

Jais-3-3.7B Reference models / Qwen3.5-4B Reference models / Gemma 4 E2B Reference models / granite-4.0-h-3b-ar Reference models / Falcon-H1-3B-Instruct
# Params 3.7B 4B 5B 3B 3B
# Activated params 3.7B 4B 2.3B 3B 3B
Architecture Dense Dense Dense Dense Dense
Open-ended Generation
AraGen
AraGen v3 (3C3H) 09-2025 49.6 45.8 39.6 27.1 13.0
Correctness 57.9 57.2 46.5 30.8 16.8
Completeness 56.9 56.3 46.4 30.4 15.3
Conciseness 18.2 10.3 11.5 14.4 3.5
Helpfulness 51.9 45.5 41.5 27.6 13.4
Honesty 55.6 49.0 45.2 28.9 13.8
Harmlessness 56.9 56.3 46.5 30.6 15.4
Knowledge & Reasoning
General Knowledge & Comprehension
Arabic MMLU* Accuracy 83.3 75.2 69.4 59.8 51.0
Belebele Accuracy 91.9 84.0 79.6 61.9 57.9
STEM & Math
AraSTEM* Accuracy 91.7 91.4 85.1 66.5 64.9
Arabic GSM8K* Accuracy 91.9 88.7 81.7 67.2 72.2
GATmath* Accuracy 96.2 95.9 90.5 45.3 41.3
Professional Domains
MedAraBench* Accuracy 74.1 74.1 57.2 38.1 34.5
Arabic Legal (HELM)* Accuracy 66.7 63.3 57.1 56.1 53.0
Arabic Language & Culture
Islamic Knowledge & Arab Culture
IslamicQA Accuracy (mean of the 4 below) 81.7 61.6 63.0 41.2 48.2
PaLM-X 2025-2* Accuracy 88.9 79.6 79.2 57.5 66.6
QIAS-1* Accuracy 83.3 50.7 51.6 18.2 27.5
QIAS-2* Accuracy 80.0 69.9 65.3 66.5 54.3
IslamicEval2025-1B* Accuracy 74.5 46.3 55.8 22.5 44.6
Arab Culture Accuracy (mean of the 3 below) 81.2 68.8 66.8 60.2 54.8
ArabCulture* Accuracy 84.3 76.2 74.7 68.4 64.2
AraDiCE-Culture* Accuracy 76.5 61.5 56.4 53.6 45.8
Arabic Cuisine^ Accuracy 82.6 68.9 69.2 58.6 54.4
Dialects
Dialectal Arabic MMLU* Accuracy 81.6 76.5 65.1 50.7 43.8
Dialect Identification (QADI) Accuracy 68.8 37.4 44.4 42.2 28.5
Alyah* Accuracy 79.7 63.3 69.7 66.0 58.9
Linguistics & Literature
Arabic MMLU with diacritics Accuracy 80.7 70.8 64.7 53.3 34.6
FannOrFlop-MCQ Accuracy 66.2 50.9 60.0 48.8 41.1
Arabic Linguistics^ Accuracy (mean of the 2 below) 92.1 72.7 77.9 49.3 42.1
Diacritization Accuracy 86.9 79.2 74.2 64.1 51.2
Parsing Accuracy 97.3 66.3 81.6 34.6 32.9
Assistant Capabilities
Instruction Following
Arabic IFBench^ Accuracy (prompt-level strict) 92.6 53.7 37.6 14.7 18.4
Arabic IFEval Accuracy 79.0 73.0 69.6 43.1 55.7
Factuality & Hallucination
HalluScore Accuracy 48.3 37.5 33.3 16.4 13.2
SimpleQA Verified (Ar) Aggregate (2/3 F-score + 1/3 non-hallucination rate) 29.1 7.0 29.8 3.5 2.5
F-score 1.5 7.4 2.2 4.5 2.3
Non-hallucination rate 84.2 6.3 84.8 1.4 3.0
Agentic
Terminal Bench 2.1 Accuracy (Avg@3) 41.2 25.8 0.4 0.0 0.0
GDPVal Accuracy (@ 75% threshold) 54.5 15.0 3.2 0.0 0.0
MCPMark Accuracy (Pass@1) 30.7 25.2 1.6 0.0 0.0
WildClawBench Mean Score@3 40.9 33.3 18.2 8.2 0.1
Apex-Agents Accuracy (Pass@1) 16.7 6.4 0.0 0.5 0.2
GAIA v2 lit AR Accuracy (Pass@1) 45.5 27.9 18.2 0.0 0.0

Looking at the published measurement results, the model outperforms comparison models in Arabic MMLU (83.3% accuracy), which measures general knowledge across 57 categories of 4-choice questions, and Arabic GSM8K (91.9% accuracy), which solves elementary school-level math problems. It also records high figures in agent-related and instruction-following metrics, such as Arabic IFEval (79.0%) measuring formal instruction compliance, Terminal Bench 2.1 (41.2%) measuring the ability to execute commands and perform tasks in a terminal, and GAIA v2 lit AR (45.5%) measuring the ability to solve research tasks.

On the other hand, in SimpleQA Verified (Ar), which checks for fabricated information in short fact-based questions with a single correct answer, while the non-hallucination rate is high at 84.2%, the F-score remains at 1.5%, falling below Qwen3.5-4B’s 7.4% and granite-4.0-h-3b-ar’s 4.5%. While it shows strengths in Arabic-world knowledge and agent tasks overall, it reveals that a balance between precision and recall in specific short fact-checking tasks remains a challenge.

Strengths and Use Cases

Jais-3-3.7B is developed for conversational assistants targeting Arabic and English, as well as agent use cases that autonomously call external tools to perform tasks. Despite being a small-scale 3.7B class model, it features an ultra-long context length of up to 512K tokens, making it suitable for reading long texts and conducting dialogues that maintain complex contexts.

According to the model card, specific areas of strength and operational tips for this model include the following:

  • Separation of Advanced Reasoning and Thought Processes: It features a thinking mode that performs logical progression inside the model, allowing API responses to output the thought content (reasoning_content) and final answer (content) separately. The publisher recommends specifying reasoning_effort="high" as a per-request parameter to fully unleash its reasoning performance.
  • Tool Calling and Agent Capabilities: It has the capability to autonomously perform function calling. In addition to the default xml, the output format for tool calls also supports json and xml_typed, which can be switched via settings.
  • Language-Specific Recommended Generation Parameters: For sampling settings when using the thinking mode, temperature=1.0 is recommended for English and temperature=0.7 for Arabic (with temperature=0.2 for default use, and a common top_p=0.95 and top_k=20).

Meanwhile, it is explicitly stated that usage in languages other than Arabic and English is out of scope. Furthermore, because it may contain outputs with content differing from facts (hallucinations) or biases derived from training data, it is considered unsuitable for critical judgment applications such as medical, legal, or financial use without human supervision.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch) — 5.1B parameters (taken from the base model inception42/Jais-3-3.7B)

Your VRAM Quantization File size Est. memory needed
4GB (laptop iGPU / phone class) Q5_0 3.3GB 4.0GB
8GB (RTX 4060 / 3060 Ti, etc.) Q8_0 5.0GB 6.0GB
12GB (RTX 4070 / 3060 12GB, etc.) BF16 9.4GB 11.3GB

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

Runs in Ollama, LM Studio and llama.cpp as-is.

It is distributed in GGUF, so no conversion is needed.

License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.

Compression: the Q5_0 build measures 5.6 bits per weight — about 35% the size of the original 16-bit weights, calculated by this site from the actual file sizes.

Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Recent Models in the Same Size Class

Models with 4–15B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.

→ Scroll horizontally to see all columns

Model Parameters Smallest VRAM tier License Our article
microsoft/UserRepGen-4B 4.5B 4GB mit UserRepGen-4B Text Generation Model: 4GB+ VRAM, GGUF Builds (2026-11-17)
IFM/Jais-3-7B-GGUF 9.0B 8GB apache-2.0 Jais-3-7B-GGUF Arabic and English 7B Reasoning Model: 8GB+ VRAM (2026-11-17)
ibm-granite/granite-switch-4.1-8b 8.8B 24GB apache-2.0 IBM Releases Granite Switch 4.1: Modular LLMs with aLoRA and Speech (2026-11-17)
ggml-org/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF 8.0B 12GB apache-2.0 NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF Text Generation Model: 12GB+ VRAM (2026-11-17)
ByteDance-Seed/Cola2 8.8B 48GB — ByteDance-Seed Releases Cola2 Language Model (2026-11-17)

How to Get It

Because the model repository is not gated, it can be downloaded without prior application via a Hugging Face account or agreement to terms of use. The license is Apache 2.0.

Distribution Formats

In this repository (IFM/Jais-3-3.7B-GGUF), files quantized in GGUF format for llama.cpp are distributed. The available variants are as follows:

  • BF16
  • Q8_0
  • Q6_K
  • Q5_K_M
  • Q5_0
  • Q4_K_M

Additionally, the collection separately provides FP8 formats and MLX formats (4bit, 8bit, bf16) tailored for Apple Silicon environments.

Inference Engine and Startup Method

Support for the K2 Horizon architecture adopted by this model has been merged into a llama.cpp pull request (ggml-org/llama.cpp#29535). Consequently, running it requires using a recent llama.cpp build (such as the master branch) that supports K2 Horizon.

An example command to build in a CUDA environment and start llama-server is as follows:

cmake -B build -DGGML_CUDA=ON
cmake --build build -j --target llama-server
./build/bin/llama-server -hf IFM/Jais-3-3.7B-GGUF:Q8_0 --jinja -ngl 99 -c 32768

The startup option -hf allows directly fetching and executing variants such as :Q8_0 or :Q4_K_M from the choices above. Additionally, to correctly process the thought process and tool-calling templates, it is recommended to pass the --jinja option when launching the server.

Example Usage via API

Here is an implementation example accessing the launched server from an OpenAI-compatible Python client. You can execute inference including the thought process.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="Jais-3-3.7B",
    messages=[
        {
            "role": "user",
            "content": "لدينا عددان مجموعهما 50 وحاصل ضربهما 600. ما هما العددان؟ اشرح خطوات الحل。",
        }
    ],
    temperature=0.7,
    top_p=0.95,
    max_tokens=32768,
    extra_body={
        "top_k": 20,
        "chat_template_kwargs": {"reasoning_effort": "high"},
    },
)

message = response.choices[0].message
print("Reasoning:", getattr(message, "reasoning_content", None))
print("Answer:", message.content)

Related Articles

What to Read Next

Sources