Qwen3.5-4B-GGUF Vision-Language Model: 4GB+ VRAM

Qwen3.5-4B-GGUF Vision-Language Model: 4GB+ VRAM

At a Glance

Item Value
Repository ggml-org/Qwen3.5-4B-GGUF
Family guide Qwen3.5 guide (8 articles)
Publisher guide Alibaba (Qwen): models and licenses
Published 2026-11-12
License apache-2.0
Formats GGUF
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

ggml-org has released the GGUF quantized repository ggml-org/Qwen3.5-4B-GGUF for the multimodal language model “Qwen3.5-4B". This repository is an automated GGUF conversion of the 4B parameter model “Qwen3.5-4B" developed by Alibaba using an automated conversion tool. In addition to text generation, the model supports image and video inputs, making it easy to handle in local environments.

Specifications

  • Number of parameters: 4B
  • Architecture: Qwen3_5ForConditionalGeneration (hybrid configuration combining Gated Delta Networks and sparse Mixture-of-Experts)
  • Hidden layer dimension: 2560
  • Number of layers: 32
  • Context length: 262,144 tokens by default (expandable up to 1,010,000 tokens)

Performance

Shown below are the evaluation results published in the model card of the source model “Qwen3.5-4B". The comparison table focuses on models that are easy to run locally, with some columns filtered out.

Language Performance

→ Scroll horizontally to see all columns

GPT-OSS-20B Qwen3-30BA3B-Thinking-2507 Qwen3.5-9B Qwen3.5-4B
Knowledge & STEM
MMLU-Pro 74.8 80.9 82.5 79.1
MMLU-Redux 87.8 91.4 91.1 88.8
C-Eval 71.4 87.4 88.2 85.1
SuperGPQA 48.5 56.8 58.2 52.9
GPQA Diamond 71.5 73.4 81.7 76.2
Instruction Following
IFEval 88.2 88.9 91.5 89.8
IFBench 65.1 51.5 64.5 59.2
MultiChallenge 40.1 46.5 54.5 49.0
Long Context
AA-LCR 30.7 49.0 63.0 57.0
LongBench v2 45.6 44.8 55.2 50.0
Reasoning & Coding
HMMT Feb 25 76.7 63.1 83.2 74.0
HMMT Nov 25 81.8 73.8 82.9 76.8
LiveCodeBench v6 74.6 66.0 65.6 55.8
OJBench 36.3 25.1 29.2 24.1
General Agent
BFCL-V4 — 42.4 66.1 50.3
TAU2-Bench — 41.9 79.1 79.9
VITA-Bench — 14.1 29.8 22.0
DeepPlanning — 4.9 18.0 17.6
Multilingualism
MMMLU 69.7 78.4 81.2 76.1
MMLU-ProX 67.3 69.1 76.3 71.5
NOVA-63 48.7 52.5 55.9 54.3
INCLUDE 65.3 74.4 75.6 71.0
Global PIQA 79.8 80.2 83.2 78.9
PolyMATH 30.9 52.6 57.3 51.1
WMT24++ 67.8 69.3 72.6 66.6
MAXIFE 80.1 77.4 83.4 78.0

Vision-Language Performance

→ Scroll horizontally to see all columns

Gemini-2.5-Flash-Lite Qwen3-VL-30B-A3B Qwen3.5-9B Qwen3.5-4B
STEM and Puzzle
MMMU 73.4 76.0 78.4 77.6
MMMU-Pro 59.7 63.0 70.1 66.3
MathVision 52.1 65.7 78.9 74.6
Mathvista(mini) 72.8 81.9 85.7 85.1
We-Math 32.1 70.0 75.2 75.4
DynaMath 69.9 80.1 83.6 83.3
ZEROBench 1.0 0.0 3.0 3.0
ZEROBench_sub 19.2 23.7 31.1 26.3
VlmsAreBlind 68.4 72.5 93.7 92.6
BabyVision 17.5 18.6 28.6/25.8 16.0/19.1
General VQA
RealWorldQA 72.2 77.4 80.3 79.5
MMStar 69.1 75.5 79.7 78.3
MMBench EN-DEV-v1.1 82.7 88.9 90.1 89.4
SimpleVQA 54.1 54.3 51.2 43.4
HallusionBench 64.5 66.0 69.3 65.0
Text Recognition and Document Understanding
OmniDocBench1.5 79.4 86.8 87.7 86.2
CharXiv(RQ) 56.1 56.6 73.0 70.8
MMLongBench-Doc 46.5 47.4 57.7 54.2
CC-OCR 72.9 77.8 79.3 76.7
AI2D_TEST 85.7 86.9 90.2 89.6
OCRBench 82.5 83.9 89.2 85.0
Spatial Intelligence
ERQA 44.3 45.3 55.5 54.0
CountBench 79.2 90.0 97.2 96.3
RefCOCO(avg) — 89.3 89.7 88.1
EmbSpatialBench 66.1 80.6 83.0 81.3
RefSpatialBench 11.2 54.2 58.5 54.6
LingoQA 17.8 62.0 80.4 74.4
Hypersim — 11.4 13.5 12.5
Nuscene — 10.3 11.8 9.9
Video Understanding
VideoMME (w sub.) 74.6 79.9 84.5 83.5
VideoMME (w/o sub.) 72.7 73.3 78.4 76.9
VideoMMMU 69.2 75.0 78.9 74.1
MLVU 78.5 78.9 84.4 82.8
MVBench — 72.0 74.4 71.2
LVBench 60.9 59.2 70.0 66.4
MMVU 65.3 66.1 67.8 64.9
Visual Agent
ScreenSpot Pro — 60.5 65.2 60.3
OSWorld-Verified — 30.6 41.8 35.6
AndroidWorld — 55.0 57.8 58.6
Tool Calling
TIR-Bench 21.5 22.5 45.6/31.9 38.9/29.9
V* 69.6 83.2 90.1/88.5 84.3/86.4
Medical VQA
SLAKE 65.0 68.8 79.0 76.1
PMC-VQA 48.8 51.5 57.9 55.5
MedXpertQA-MM 35.3 35.5 49.9 42.9

According to the evaluation results of the source model published in the model card, Qwen3.5-4B demonstrates high overall capability despite being a relatively small model in the 4B class. Notably, it achieves 89.8 on IFEval, which measures instruction-following performance, 85.0 on OCRBench, which measures text reading capability in images, and records 77.6 on MMMU, which evaluates academic-level image understanding, showing strengths in instruction compliance and visual tasks that rival large models. On the other hand, it scores 55.8 on LiveCodeBench v6, which handles competitive programming coding problems, and 35.6 on OSWorld-Verified, which measures OS screen operation capability, indicating that compared to larger models like Qwen3.5-9B, it falls short in complex programming and advanced GUI operation agent domains.

Strengths and Use Cases

The strengths and expected use cases of this model are based on the capabilities of the source model “Qwen3.5-4B". It is suitable for the following uses and tasks:

  • Multimodal processing including images and videos: Since it can integrate and process not only text but also image and video inputs, it can be utilized for reading charts and graphs, document understanding, visual question answering, and video summarization.
  • Long text analysis: It supports a standard context length of 262,144 tokens, enabling the reading of lengthy documents. Additionally, by applying RoPE scaling techniques such as YaRN, it can be extended and processed up to 1,010,000 tokens.
  • Multilingual dialogue and task execution: It supports 201 languages and dialects, accommodating translation, question answering, and dialogue incorporating regional nuances in multilingual environments.
  • Tool calling and agent operations: Equipped with tool calling capabilities, it can perform file system operations and task automation in conjunction with MCP (Model Context Protocol) servers through Qwen-Agent or Qwen Code, an AI agent for the terminal.

Note that Qwen3.5-4B operates by default in Thinking Mode, which outputs thought processes, but by specifying enable_thinking: False in the API parameters, it can also be switched to Instruct Mode to obtain direct answers without stepping through thoughts.

How It Differs from Similar Models

Tev1-4B-experimental: A 4B Model Specialized for Structured Data Decision-Making Requiring 4GB+ VRAM, previously covered on our site, is a derivative model obtained by applying supervised fine-tuning (SFT) using Qwen3.5-4B as the base model.

Tev1-4B-experimental is specialized for outputting a single choice label from structured inputs (states, questions, choices) and was developed to streamline decision-making and classification tasks. In contrast, ggml-org/Qwen3.5-4B-GGUF released this time retains the source model’s general-purpose text dialogue capabilities, image/video comprehension, long-text reading, and broad multimodal functions such as tool calling, while being converted into the GGUF format for ease of running locally.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch) — 4.7B parameters (taken from the base model Qwen/Qwen3.5-4B)

Your VRAM Quantization File size Est. memory needed
4GB (laptop iGPU / phone class) Q4_K_M 2.8GB 3.4GB
8GB (RTX 4060 / 3060 Ti, etc.) Q8_0 4.2GB 5.0GB
12GB (RTX 4070 / 3060 12GB, etc.) BF16 7.8GB 9.4GB

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

Runs in Ollama, LM Studio and llama.cpp as-is.

It is distributed in GGUF, so no conversion is needed.

License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.

Compression: the Q4_K_M build measures 5.16 bits per weight — about 32% the size of the original 16-bit weights, calculated by this site from the actual file sizes.

Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Recent Models in the Same Size Class

Models with 4–15B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.

→ Scroll horizontally to see all columns

Model Parameters Smallest VRAM tier License Our article
tencent/Youtu-Parsing-Omni 5.3B 12GB youtu-parsing Youtu-Parsing-Omni Omnimodal Analysis Model: 12GB+ VRAM (2026-11-17)
nvidia/Muse-Spark-Open-816B-A42B-DFlash2 4.4B 12GB nvidia-open-model-agreement-license Muse-Spark-Open-816B-A42B-DFlash2 Vision-Language Model: 12GB+ VRAM (2026-11-17)
tencent/WeVisOne-VL-4B 4.8B 4GB apache-2.0 Tencent Releases WeVisOne-VL Vision Language Models (2026-11-17)
inclusionAI/ZwZ2-9B-GGUF 9.4B 8GB apache-2.0 ZwZ2-9B-GGUF Vision-Language Model: 8GB+ VRAM (2026-11-17)
microsoft/Mage-VL-Cowork 4.7B 12GB apache-2.0 Mage-VL-Cowork Vision-Language Model: 12GB+ VRAM, GGUF Builds (2026-11-17)

How to Get It

This model is distributed in GGUF format and can be obtained from the Hugging Face repository ggml-org/Qwen3.5-4B-GGUF. The Apache 2.0 license applies, and prior license agreement procedures (Gated) are not required for use.

For verification and use, Llama app is supported, and you can also run the following command from the terminal to load the model and start the server:

llama serve -hf ggml-org/Qwen3.5-4B-GGUF

Note that the model in this repository is a quantized version automatically generated using an automated conversion tool ( https://github.com/ggml-org/convert ).

Related Articles

What to Read Next

Sources