Qwen3.5-4B-GGUF Vision-Language Model: 4GB+ VRAM

At a Glance
| Item | Value |
|---|---|
| Repository | ggml-org/Qwen3.5-4B-GGUF |
| Family guide | Qwen3.5 guide (8 articles) |
| Publisher guide | Alibaba (Qwen): models and licenses |
| Published | 2026-11-12 |
| License | apache-2.0 |
| Formats | GGUF |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
ggml-org has released the GGUF quantized repository ggml-org/Qwen3.5-4B-GGUF for the multimodal language model “Qwen3.5-4B". This repository is an automated GGUF conversion of the 4B parameter model “Qwen3.5-4B" developed by Alibaba using an automated conversion tool. In addition to text generation, the model supports image and video inputs, making it easy to handle in local environments.
Specifications
- Number of parameters: 4B
- Architecture: Qwen3_5ForConditionalGeneration (hybrid configuration combining Gated Delta Networks and sparse Mixture-of-Experts)
- Hidden layer dimension: 2560
- Number of layers: 32
- Context length: 262,144 tokens by default (expandable up to 1,010,000 tokens)
Performance
Shown below are the evaluation results published in the model card of the source model “Qwen3.5-4B". The comparison table focuses on models that are easy to run locally, with some columns filtered out.
Language Performance
→ Scroll horizontally to see all columns
| GPT-OSS-20B | Qwen3-30BA3B-Thinking-2507 | Qwen3.5-9B | Qwen3.5-4B | |
|---|---|---|---|---|
| Knowledge & STEM | ||||
| MMLU-Pro | 74.8 | 80.9 | 82.5 | 79.1 |
| MMLU-Redux | 87.8 | 91.4 | 91.1 | 88.8 |
| C-Eval | 71.4 | 87.4 | 88.2 | 85.1 |
| SuperGPQA | 48.5 | 56.8 | 58.2 | 52.9 |
| GPQA Diamond | 71.5 | 73.4 | 81.7 | 76.2 |
| Instruction Following | ||||
| IFEval | 88.2 | 88.9 | 91.5 | 89.8 |
| IFBench | 65.1 | 51.5 | 64.5 | 59.2 |
| MultiChallenge | 40.1 | 46.5 | 54.5 | 49.0 |
| Long Context | ||||
| AA-LCR | 30.7 | 49.0 | 63.0 | 57.0 |
| LongBench v2 | 45.6 | 44.8 | 55.2 | 50.0 |
| Reasoning & Coding | ||||
| HMMT Feb 25 | 76.7 | 63.1 | 83.2 | 74.0 |
| HMMT Nov 25 | 81.8 | 73.8 | 82.9 | 76.8 |
| LiveCodeBench v6 | 74.6 | 66.0 | 65.6 | 55.8 |
| OJBench | 36.3 | 25.1 | 29.2 | 24.1 |
| General Agent | ||||
| BFCL-V4 | — | 42.4 | 66.1 | 50.3 |
| TAU2-Bench | — | 41.9 | 79.1 | 79.9 |
| VITA-Bench | — | 14.1 | 29.8 | 22.0 |
| DeepPlanning | — | 4.9 | 18.0 | 17.6 |
| Multilingualism | ||||
| MMMLU | 69.7 | 78.4 | 81.2 | 76.1 |
| MMLU-ProX | 67.3 | 69.1 | 76.3 | 71.5 |
| NOVA-63 | 48.7 | 52.5 | 55.9 | 54.3 |
| INCLUDE | 65.3 | 74.4 | 75.6 | 71.0 |
| Global PIQA | 79.8 | 80.2 | 83.2 | 78.9 |
| PolyMATH | 30.9 | 52.6 | 57.3 | 51.1 |
| WMT24++ | 67.8 | 69.3 | 72.6 | 66.6 |
| MAXIFE | 80.1 | 77.4 | 83.4 | 78.0 |
Vision-Language Performance
→ Scroll horizontally to see all columns
| Gemini-2.5-Flash-Lite | Qwen3-VL-30B-A3B | Qwen3.5-9B | Qwen3.5-4B | |
|---|---|---|---|---|
| STEM and Puzzle | ||||
| MMMU | 73.4 | 76.0 | 78.4 | 77.6 |
| MMMU-Pro | 59.7 | 63.0 | 70.1 | 66.3 |
| MathVision | 52.1 | 65.7 | 78.9 | 74.6 |
| Mathvista(mini) | 72.8 | 81.9 | 85.7 | 85.1 |
| We-Math | 32.1 | 70.0 | 75.2 | 75.4 |
| DynaMath | 69.9 | 80.1 | 83.6 | 83.3 |
| ZEROBench | 1.0 | 0.0 | 3.0 | 3.0 |
| ZEROBench_sub | 19.2 | 23.7 | 31.1 | 26.3 |
| VlmsAreBlind | 68.4 | 72.5 | 93.7 | 92.6 |
| BabyVision | 17.5 | 18.6 | 28.6/25.8 | 16.0/19.1 |
| General VQA | ||||
| RealWorldQA | 72.2 | 77.4 | 80.3 | 79.5 |
| MMStar | 69.1 | 75.5 | 79.7 | 78.3 |
| MMBench EN-DEV-v1.1 | 82.7 | 88.9 | 90.1 | 89.4 |
| SimpleVQA | 54.1 | 54.3 | 51.2 | 43.4 |
| HallusionBench | 64.5 | 66.0 | 69.3 | 65.0 |
| Text Recognition and Document Understanding | ||||
| OmniDocBench1.5 | 79.4 | 86.8 | 87.7 | 86.2 |
| CharXiv(RQ) | 56.1 | 56.6 | 73.0 | 70.8 |
| MMLongBench-Doc | 46.5 | 47.4 | 57.7 | 54.2 |
| CC-OCR | 72.9 | 77.8 | 79.3 | 76.7 |
| AI2D_TEST | 85.7 | 86.9 | 90.2 | 89.6 |
| OCRBench | 82.5 | 83.9 | 89.2 | 85.0 |
| Spatial Intelligence | ||||
| ERQA | 44.3 | 45.3 | 55.5 | 54.0 |
| CountBench | 79.2 | 90.0 | 97.2 | 96.3 |
| RefCOCO(avg) | — | 89.3 | 89.7 | 88.1 |
| EmbSpatialBench | 66.1 | 80.6 | 83.0 | 81.3 |
| RefSpatialBench | 11.2 | 54.2 | 58.5 | 54.6 |
| LingoQA | 17.8 | 62.0 | 80.4 | 74.4 |
| Hypersim | — | 11.4 | 13.5 | 12.5 |
| Nuscene | — | 10.3 | 11.8 | 9.9 |
| Video Understanding | ||||
| VideoMME (w sub.) | 74.6 | 79.9 | 84.5 | 83.5 |
| VideoMME (w/o sub.) | 72.7 | 73.3 | 78.4 | 76.9 |
| VideoMMMU | 69.2 | 75.0 | 78.9 | 74.1 |
| MLVU | 78.5 | 78.9 | 84.4 | 82.8 |
| MVBench | — | 72.0 | 74.4 | 71.2 |
| LVBench | 60.9 | 59.2 | 70.0 | 66.4 |
| MMVU | 65.3 | 66.1 | 67.8 | 64.9 |
| Visual Agent | ||||
| ScreenSpot Pro | — | 60.5 | 65.2 | 60.3 |
| OSWorld-Verified | — | 30.6 | 41.8 | 35.6 |
| AndroidWorld | — | 55.0 | 57.8 | 58.6 |
| Tool Calling | ||||
| TIR-Bench | 21.5 | 22.5 | 45.6/31.9 | 38.9/29.9 |
| V* | 69.6 | 83.2 | 90.1/88.5 | 84.3/86.4 |
| Medical VQA | ||||
| SLAKE | 65.0 | 68.8 | 79.0 | 76.1 |
| PMC-VQA | 48.8 | 51.5 | 57.9 | 55.5 |
| MedXpertQA-MM | 35.3 | 35.5 | 49.9 | 42.9 |
According to the evaluation results of the source model published in the model card, Qwen3.5-4B demonstrates high overall capability despite being a relatively small model in the 4B class. Notably, it achieves 89.8 on IFEval, which measures instruction-following performance, 85.0 on OCRBench, which measures text reading capability in images, and records 77.6 on MMMU, which evaluates academic-level image understanding, showing strengths in instruction compliance and visual tasks that rival large models. On the other hand, it scores 55.8 on LiveCodeBench v6, which handles competitive programming coding problems, and 35.6 on OSWorld-Verified, which measures OS screen operation capability, indicating that compared to larger models like Qwen3.5-9B, it falls short in complex programming and advanced GUI operation agent domains.
Strengths and Use Cases
The strengths and expected use cases of this model are based on the capabilities of the source model “Qwen3.5-4B". It is suitable for the following uses and tasks:
- Multimodal processing including images and videos: Since it can integrate and process not only text but also image and video inputs, it can be utilized for reading charts and graphs, document understanding, visual question answering, and video summarization.
- Long text analysis: It supports a standard context length of 262,144 tokens, enabling the reading of lengthy documents. Additionally, by applying RoPE scaling techniques such as YaRN, it can be extended and processed up to 1,010,000 tokens.
- Multilingual dialogue and task execution: It supports 201 languages and dialects, accommodating translation, question answering, and dialogue incorporating regional nuances in multilingual environments.
- Tool calling and agent operations: Equipped with tool calling capabilities, it can perform file system operations and task automation in conjunction with MCP (Model Context Protocol) servers through Qwen-Agent or Qwen Code, an AI agent for the terminal.
Note that Qwen3.5-4B operates by default in Thinking Mode, which outputs thought processes, but by specifying enable_thinking: False in the API parameters, it can also be switched to Instruct Mode to obtain direct answers without stepping through thoughts.
How It Differs from Similar Models
Tev1-4B-experimental: A 4B Model Specialized for Structured Data Decision-Making Requiring 4GB+ VRAM, previously covered on our site, is a derivative model obtained by applying supervised fine-tuning (SFT) using Qwen3.5-4B as the base model.
Tev1-4B-experimental is specialized for outputting a single choice label from structured inputs (states, questions, choices) and was developed to streamline decision-making and classification tasks. In contrast, ggml-org/Qwen3.5-4B-GGUF released this time retains the source model’s general-purpose text dialogue capabilities, image/video comprehension, long-text reading, and broad multimodal functions such as tool calling, while being converted into the GGUF format for ease of running locally.
Hardware Requirements
Estimated requirements (calculated by Local Model Watch) — 4.7B parameters (taken from the base model Qwen/Qwen3.5-4B)
| Your VRAM | Quantization | File size | Est. memory needed |
|---|---|---|---|
| 4GB (laptop iGPU / phone class) | Q4_K_M | 2.8GB | 3.4GB |
| 8GB (RTX 4060 / 3060 Ti, etc.) | Q8_0 | 4.2GB | 5.0GB |
| 12GB (RTX 4070 / 3060 12GB, etc.) | BF16 | 7.8GB | 9.4GB |
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.
Can You Run It Locally?
Runs in Ollama, LM Studio and llama.cpp as-is.
It is distributed in GGUF, so no conversion is needed.
License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.
Compression: the Q4_K_M build measures 5.16 bits per weight — about 32% the size of the original 16-bit weights, calculated by this site from the actual file sizes.
Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.
Recent Models in the Same Size Class
Models with 4–15B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.
→ Scroll horizontally to see all columns
| Model | Parameters | Smallest VRAM tier | License | Our article |
|---|---|---|---|---|
| tencent/Youtu-Parsing-Omni | 5.3B | 12GB | youtu-parsing | Youtu-Parsing-Omni Omnimodal Analysis Model: 12GB+ VRAM (2026-11-17) |
| nvidia/Muse-Spark-Open-816B-A42B-DFlash2 | 4.4B | 12GB | nvidia-open-model-agreement-license | Muse-Spark-Open-816B-A42B-DFlash2 Vision-Language Model: 12GB+ VRAM (2026-11-17) |
| tencent/WeVisOne-VL-4B | 4.8B | 4GB | apache-2.0 | Tencent Releases WeVisOne-VL Vision Language Models (2026-11-17) |
| inclusionAI/ZwZ2-9B-GGUF | 9.4B | 8GB | apache-2.0 | ZwZ2-9B-GGUF Vision-Language Model: 8GB+ VRAM (2026-11-17) |
| microsoft/Mage-VL-Cowork | 4.7B | 12GB | apache-2.0 | Mage-VL-Cowork Vision-Language Model: 12GB+ VRAM, GGUF Builds (2026-11-17) |
How to Get It
This model is distributed in GGUF format and can be obtained from the Hugging Face repository ggml-org/Qwen3.5-4B-GGUF. The Apache 2.0 license applies, and prior license agreement procedures (Gated) are not required for use.
For verification and use, Llama app is supported, and you can also run the following command from the terminal to load the model and start the server:
llama serve -hf ggml-org/Qwen3.5-4B-GGUF
Note that the model in this repository is a quantized version automatically generated using an automated conversion tool ( https://github.com/ggml-org/convert ).
Related Articles
- llama.cpp Speculative Decoding Test GGUF Models Released
- ZwZ2-4B-GGUF Vision-Language Model: 4GB+ VRAM
- UserRepGen-4B Text Generation Model: 4GB+ VRAM, GGUF Builds
- Tev1-4B-experimental Text Generation Model: 4GB+ VRAM, GGUF Builds
What to Read Next
- Find models by VRAM (This model runs from the 4GB tier) → Other models that run on a 8GB GPU
- Explore the same model family → Qwen3.5 family overview (8 articles, 10 converted builds)
- Engines that run this model → llama.cpp / Ollama
- What Q4_K_M, Q8_0 mean and where to get this model → GGUF format guide and models / Quantization and model-format glossary
- Learn about the publisher → Alibaba (Qwen): models, licenses and articles

