ZwZ2-9B-GGUF Vision-Language Model: 8GB+ VRAM

ZwZ2-9B-GGUF Vision-Language Model: 8GB+ VRAM

At a Glance

Item Value
Repository inclusionAI/ZwZ2-9B-GGUF
Family guide Qwen3.5 guide (8 articles)
Publisher guide inclusionAI (Ant Group): models and licenses
Published 2026-10-30
License apache-2.0
Formats GGUF
Paper arXiv:2610.37037
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

“inclusionAI/ZwZ2-9B-GGUF", a GGUF format version of the open-weight vision-language model “ZwZ2-9B", has been released on Hugging Face. This model is a quantized version of “ZwZ2-9B", developed as part of the “Zooming without Zooming v2 (ZwZ2)" project, converted to run on engines such as llama.cpp.

According to the paper on which this model is based, “Zooming without Zooming v2: From Seeing Details to Connecting Evidence for Generalizable Multimodal Perception" (arXiv:2610.37037), the goal of the model is to strengthen the ability to “connect evidence", which underlies spatial perception and multimodal long-context understanding. This refers to the ability to identify related visual facts, link them to the correct targets, restore relationships, and construct answers. It uses full images during training, and is characterized by its ability to integrate and process visual evidence from multiple regions during inference without using external zoom tools or the like.

The distributed repository provides the language model part and the visual encoder/projector part (mmproj) separately, allowing users to combine FP16, Q8_0, and Q4_K_M precisions to suit their local environment.

Specifications

  • Parameters: 9B
  • Architecture: Qwen3_5ForConditionalGeneration (Base model: Qwen3.5-9B)

Performance

Performance data for this model has been published in the model card and paper of the quantization source model, “inclusionAI/ZwZ2-9B". As mentioned in the quantized model card, please note that applying quantization may affect accuracy.

According to the measurement results described in the original model card, accuracy improvements over the foundational Qwen3.5-9B have been confirmed across a total of 25 benchmarks spanning 6 domains. Specifically, the overall accuracy is reported to have improved by 4.96 points from the base model’s 51.63% to 56.59%. In addition, it is stated that it surpassed the base model’s scores in all 25 of these benchmarks.

This improvement is described as extending not only to specific tasks, but also to fine-grained perception, spatial reasoning, long-context understanding, multimodal agents, and general visual reasoning. Note that “MMLongBench*" used in the evaluation of long-context understanding was tested on a total of 25 non-counting tasks consisting of V-RAG, Image MM-NIAH, and VH-Multi.

These scores are actual measurement results published by the model developers themselves, rather than independent benchmarks by third parties. The original model card does not include tabular text summarizing detailed individual scores for each benchmark; instead, the average growth and improvement trends across domains are shown both qualitatively and quantitatively.

Strengths and Use Cases

This model inherits the characteristics of the original ZwZ2-9B and excels at general tasks where visual evidence scattered across multiple regions within an image is interconnected to make judgments.

Specifically, it is suited for multi-stage visual reasoning tasks such as comparing and cross-referencing different parts of an image, or aggregating information from multiple locations to draw conclusions. In addition to grasping fine details (fine-grained perception) that are often overlooked by general vision-language models, it is expected to be utilized in areas such as spatial reasoning for grasping positional relationships between objects, task execution by multimodal agents, and embodied spatial reasoning intended for robotics.

Another feature is that improvements in “multimodal long-context understanding" are demonstrated in the paper and model card. This makes it suitable for contexts where images and text are mixed, or for use cases that maintain and integrate multiple facts read from images over a long context. Furthermore, because it can capture the relationships of each region solely from the input full image without requiring external zoom tools to crop and reload parts of the image during inference, it is suitable for systems that want to perform image understanding with a simple pipeline without inserting additional inference steps.

The training used the “EvidenceConnect" framework, which generates questions that compare and aggregate 3 to 7 related regions extracted from the SA-1B dataset. Through reinforcement learning using GRPO (Group Relative Policy Optimization) with this training data, general visual reasoning capabilities have been boosted. Main intended uses include dialogues with image inputs and visual question answering (VQA) where evidence from multiple locations is cross-examined to provide answers.

How It Differs from Similar Models

As another model developed based on Qwen3.5-9B, there is “clef-flash" (Structured output specialized model: Answer examples on this site, Required VRAM 4GB+, GGUF version available). While clef-flash is equipped with a proprietary Joint schema head and focuses on text processing performance such as decision making and schema-compliant data extraction, ZwZ2-9B is a multimodal model equipped with a visual encoder, differing in its approach by focusing primarily on the connection of visual evidence within images and spatial reasoning.

Additionally, as a model aiming for long-context processing, there is “LensVLM-9B" (Vision-language model that reduces long text as images for processing: Required VRAM 4GB+, GGUF version available). LensVLM-9B adopted an approach of rendering long text as low-resolution images and selectively expanding necessary parts to reduce the number of tokens. In contrast, ZwZ2-9B is trained with the objective of directly comparing and integrating visual evidence from multiple regions contained within a single full image, without relying on tools for zooming or resolution changing during inference.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch) — 9.4B parameters (taken from the base model inclusionAI/ZwZ2-9B)

Your VRAM Quantization File size Est. memory needed
8GB (RTX 4060 / 3060 Ti, etc.) Q4_K_M 5.2GB 6.3GB
12GB (RTX 4070 / 3060 12GB, etc.) Q8_0 8.9GB 10.6GB
24GB (RTX 4090 / 3090, etc.) F16 16.7GB 20.0GB

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

Runs in Ollama, LM Studio and llama.cpp as-is.

It is distributed in GGUF, so no conversion is needed.

License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.

Compression: the Q4_K_M build measures 4.75 bits per weight — about 30% the size of the original 16-bit weights, calculated by this site from the actual file sizes.

Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Recent Models in the Same Size Class

Models with 4–15B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.

→ Scroll horizontally to see all columns

Model Parameters Smallest VRAM tier License Our article
tencent/Youtu-Parsing-Omni 5.3B 12GB youtu-parsing Youtu-Parsing-Omni Omnimodal Analysis Model: 12GB+ VRAM (2026-11-17)
nvidia/Muse-Spark-Open-816B-A42B-DFlash2 4.4B 12GB nvidia-open-model-agreement-license Muse-Spark-Open-816B-A42B-DFlash2 Vision-Language Model: 12GB+ VRAM (2026-11-17)
tencent/WeVisOne-VL-4B 4.8B 4GB apache-2.0 Tencent Releases WeVisOne-VL Vision Language Models (2026-11-17)
inclusionAI/ZwZ2-4B-GGUF 4.5B 4GB apache-2.0 ZwZ2-4B-GGUF Vision-Language Model: 4GB+ VRAM (2026-11-17)
microsoft/Mage-VL-Cowork 4.7B 12GB apache-2.0 Mage-VL-Cowork Vision-Language Model: 12GB+ VRAM, GGUF Builds (2026-11-17)

How to Get It

This model is distributed in GGUF format and can be obtained from the Hugging Face repository “inclusionAI/ZwZ2-9B-GGUF". It is licensed under Apache-2.0 and is not a gated model requiring prior agreement to terms of use, so files can be downloaded and used directly.

The repository provides the language model itself and the visual encoder/projector (mmproj) as separate files. The provided precisions are as follows:

  • Language Model: FP16, Q8_0, Q4_K_M
  • Visual Component (mmproj): FP16, Q8_0, Q4_K_M

During execution, files are selected and combined from both the language model and the visual component. While combinations of different precisions are possible, the corresponding mmproj file must be specified when inputting images.

Execution requires a recent build of llama.cpp that supports Qwen3.5 multimodality.

When performing CLI inference from the command line, use llama-mtmd-cli as follows:

llama-mtmd-cli \
  -m ZwZ2-9B-Q8_0.gguf \
  --mmproj mmproj-ZwZ2-9B-Q8_0.gguf \
  --image /path/to/image.jpg \
  -p "Describe this image." \
  --temp 0 -n 2048

Alternatively, when launching an OpenAI-compatible API server or Web UI for interactive use, start it using llama-server as follows:

llama-server \
  -m ZwZ2-9B-Q8_0.gguf \
  --mmproj mmproj-ZwZ2-9B-Q8_0.gguf \
  --host 127.0.0.1 --port 8080

Depending on the precision you wish to use, replace the filename portions with the FP16 or Q4_K_M filenames.

Related Articles

What to Read Next

Sources