WeMM-Video-2B-NVFP4 Vision-Language Model: 8GB+ VRAM

At a Glance
| Item | Value |
|---|---|
| Repository | tencent/WeMM-Video-2B-NVFP4 |
| Family guide | WeMM-Video guide (2 articles) |
| Publisher guide | Tencent: models and licenses |
| Published | 2026-11-17 |
| License | apache-2.0 |
| Formats | safetensors |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
“WeMM-Video-2B-NVFP4", an NVFP4 quantized version of the multimodal large language model “WeMM-Video-2B" developed by Tencent, has been released. This model is part of the “WeMM-Video" model family, which supports both Chinese and English, and is designed for integrated understanding and reasoning of audio and video.
This model accepts diverse media inputs including text, single images, audio-only data, silent videos, and videos with audio, and outputs responses in text format. It is capable of accurately describing situations and events by combining video frames with attached audio, transcribing audio portions, and performing reasoning by integrating visual and auditory information.
In terms of architecture, it adopts a structure where the vision encoder and audio encoder convert each media into tokens, which are then processed together with timestamp-assigned video frames on the language model side alongside text. Approximately 1.2 trillion multimodal tokens were used for pre-training, while multi-teacher OPD, anti-repetition MrDPO (anti-repetition MrDPO), and specialist reinforcement learning were applied for post-training.
Specifications
- Parameters: Dense 2B (based on the description of the original model)
- Architecture: WeMMVideoForConditionalGeneration (Model type: qwen3_5)
- Context length: 8192
Performance
Neither the model card for this model nor the documentation for the original model “WeMM-Video-2B" contains quantitative benchmark score tables or numerical measurement data comparing it with other models.
Therefore, it is not possible to provide a quantitative judgment on which tasks it excels in numerically compared to other open-weight models, but the model card clearly states the operational verification results by the publishers.
In the verification of the quantized version “WeMM-Video-2B-NVFP4", it is reported that generation tests for five media types—text, image, audio, silent video, and video with audio—were executed and all completed successfully after loading the model using a patched vLLM 0.31.0 environment with ModelOpt Mixed Precision and Marlin (SM120 reference configuration) settings.
Furthermore, during the verification stage of the original model “WeMM-Video-2B" (BF16 version), it is reported that it cleared generation checks for text, images, audio, and video with audio in patched vLLM running on L20 and H20 GPUs, and also passed text generation tests in a CPU environment and direct loading tests from the Transformers library.
The performance and supported tasks of this quantized version basically inherit the capabilities of the original model, though it should be noted that slight precision changes may occur compared to the original model due to the applied NVFP4 W4A16 quantization.
Strengths and Use Cases
The tasks and expected use cases for “WeMM-Video-2B-NVFP4" are based on the multimodal processing capabilities possessed by its base model, “WeMM-Video-2B".
This model specializes in “joint audio-visual understanding and reasoning", which simultaneously processes, understands, and reasons about audio and video in two languages: Chinese and English. Specifically, it supports a variety of input formats and processing scenarios such as the following:
- Diverse Input Media Combinations: In addition to text, it is possible to directly input single still images, audio-only recordings, silent videos without audio, and video files containing audio tracks.
- Integrated Understanding and Reasoning of Video and Audio: It can analyze by combining the visual frame information of a video with its accompanying original audio track. Beyond merely describing what is happening on screen, it can perform transcriptions of spoken audio, as well as reasoning and rationalization combining visible and audible information.
- Switching Between Two Chat Modes: Two chat modes can be selected depending on the application. While the “Instruct mode" is set by default for normal conversations and direct responses, the “Thinking mode"—which generates responses through deeper reasoning steps—can be utilized by specifying
enable_thinking=True(orenable_thinking=true) in the chat template options.
Thus, it is suitable for applications such as interpreting audio dialogues in videos, content analysis of multimedia content, and Q&A systems combining visual and auditory information.
Hardware Requirements
Estimated requirements (calculated by Local Model Watch) — 3.3B parameters (taken from the base model tencent/WeMM-Video-2B)
| Your VRAM | Quantization | File size | Est. memory needed |
|---|---|---|---|
| 8GB (RTX 4060 / 3060 Ti, etc.) | NVFP4 | 4.9GB | 5.9GB |
Inference engine support (architecture name matched against each project’s own model registry in its source code, checked 2026-11-20): llama.cpp: not registered, vLLM: not registered, MLX (mlx-lm): not registered. “Not registered" means the name is absent from that registry today, not that the model cannot run.
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.
Can You Run It Locally?
Not usable in Ollama, LM Studio and llama.cpp yet.
The publisher ships safetensors only, and llama.cpp’s registry does not list this architecture. llama.cpp would need to add support before these tools can run it. Today it can be run with transformers, using the memory figures in the table above.
License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.
Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.
Recent Models in the Same Size Class
Models with up to 4B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.
→ Scroll horizontally to see all columns
| Model | Parameters | Smallest VRAM tier | License | Our article |
|---|---|---|---|---|
| LiquidAI/d1-omni-600M | 587M | 4GB | lfm1.0 | d1-omni-600M Multimodal Decision Model: 4GB+ VRAM, GGUF Builds (2026-11-17) |
| LiquidAI/d1-3B | 3.1B | 4GB | lfm1.0 | d1-3B Vision-Language Model: 4GB+ VRAM, GGUF Builds (2026-11-17) |
| baidu/Unlimited-OCR-2 | 3.3B | 8GB | mit | Unlimited-OCR-2 Vision-Language Model: 8GB+ VRAM (2026-11-17) |
| openbmb/MiniCPM-OCR-1.0-DSpark | 297M | 4GB | apache-2.0 | MiniCPM-OCR-1.0-DSpark OCR Acceleration Draft Model: 4GB+ VRAM (2026-11-17) |
| openbmb/MiniCPM-OCR-Jev-GPTQ | 2.2B | 4GB | apache-2.0 | OpenBMB Releases Quantized Versions of MiniCPM-OCR-Jev (2026-11-17) |
How to Get It
This model is published on Hugging Face under the Apache 2.0 license and can be freely downloaded and used by anyone. Prior license agreement or gated access application is not required for use.
The distribution format is Safetensors, with ModelOpt “NVFP4 W4A16" mixed-precision quantization applied. NVFP4 with a group-16 setting is used for the Feed-Forward (FFN) weights of the language model part, and BF16 is used for activations. Meanwhile, other tensors such as attention weights and normalization layers (Norm) retain the precision of the original model.
For model operation and serving, it is recommended to use the vLLM 0.31.0 environment (supporting ModelOpt Mixed Precision and Marlin backends) with the officially provided WeMM runtime patch applied.
When starting the server, run the vllm serve command as follows:
MODEL=tencent/WeMM-Video-2B-NVFP4
vllm serve "$MODEL" --served-model-name "$MODEL" --trust-remote-code --dtype bfloat16 \
--host 127.0.0.1 --port 8000 --max-model-len 8192 \
--max-num-seqs 1 --max-num-batched-tokens 8192 \
--no-enable-prefix-caching --mamba-cache-mode none --mamba-ssm-cache-dtype float32 \
--mm-processor-cache-gb 0 --skip-mm-profiling \
--limit-mm-per-prompt '{"image":4,"video":2,"audio":2}' \
--media-io-kwargs '{"video":{"fps":2,"min_frames":4,"max_frames":16,"total_pixels":262144}}' \
--attention-config '{"backend":"FLASH_ATTN","flash_attn_version":2}' \
--mm-encoder-attn-backend FLASH_ATTN \
--quantization modelopt_mixed --linear-backend marlin --moe-backend marlin
After starting the server locally on port 8000, you can use the request client script examples/chat.py included in the repository to send requests containing various media to the OpenAI-compatible chat endpoint:
MODEL=tencent/WeMM-Video-2B-NVFP4
python examples/chat.py text --model "$MODEL"
python examples/chat.py image --model "$MODEL"
python examples/chat.py audio --model "$MODEL"
python examples/chat.py video --model "$MODEL"
python examples/chat.py video-audio --model "$MODEL"
When sending videos, you can control the sampling behavior of the video clips by adjusting the fps and max_frames values in video_url.
Related Articles
- Tencent Releases WeMM-Video-2B-GGUF Quantized Model
- Tencent Releases Hy4 flash: A Cost-Effective Multimodal MoE Model
- Qwen3.8-27B-DSpine-b16 Draft Model for Speculative Decoding: 8GB+ VRAM
- Hy-Memory-30B-A3B Text Generation Model: 12GB+ VRAM, GGUF Builds
What to Read Next
- Find models by VRAM (This model runs from the 8GB tier) → Other models that run on a 8GB GPU
- Explore the same model family → WeMM-Video family overview (2 articles)
- What NVFP4 mean and where to get this model → NVFP4 / MXFP4 format guide and models / Quantization and model-format glossary
- Learn about the publisher → Tencent: models, licenses and articles
- Other models for the same task → Other vision-language models

