Intern-S2-397B-GGUF Vision-Language Model: ~102GB Memory
At a Glance
| Item | Value |
|---|---|
| Repository | bartowski/Intern-S2-397B-GGUF |
| Published | 2026-09-14 |
| License | apache-2.0 |
| Formats | GGUF |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
What we checked ourselves
- It needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26× as much Japanese fits in the same context length, and generation is faster per character.
Details and conditions are in “Our Own Measurements” below.
Overview
bartowski has released “bartowski/Intern-S2-397B-GGUF", a GGUF quantized version repository of the multimodal foundation model “Intern-S2-397B" developed by Shanghai AI Laboratory.
This repository is quantized using llama.cpp (release b10896) and includes a Multi-Token Prediction (MTP) layer to speed up inference along with projector files (mmproj) for image input.
Specifications
- Parameters: 403B (original model architecture is Qwen3_5MoeForConditionalGeneration)
- Context Length: Up to 256K tokens (during text inference evaluation of the original model), up to 64K tokens (during multimodal inference evaluation of the original model)
- License: apache-2.0
Strengths and Use Cases
The original model, “Intern-S2-397B", is a multimodal foundation model designed for scientific intelligence and long-horizon agents.
According to the model card, it adopts a visual pre-training approach that learns directly from scientific literature pages, modeling symbolic semantics and visual relationships in a shared representation space. Trained through scientific reinforcement learning tasks across over 20 domains, it excels at specialized scientific tasks such as biomolecular interaction design and material structure generation, as well as general and scientific autonomous agent tasks leveraging sandbox environments. It also supports switching between thinking modes, tool calling, and time-series data inference (such as seismic event detection and power load forecasting).
Note that this release is a quantized version converting the original model into GGUF format, and minor accuracy changes may occur due to quantization.
Hardware Requirements
Estimated requirements (calculated by Local Model Watch) — 403.4B parameters (taken from the base model internlm/Intern-S2-397B)
| Your VRAM | Quantization | File size | Est. memory needed |
|---|---|---|---|
| More than 102GB of VRAM (multi-GPU or CPU offload required) | IQ1_S | 85.3GB | 102.4GB |
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.
Can You Run It Locally?
Runs in Ollama, LM Studio and llama.cpp as-is.
It is distributed in GGUF, so no conversion is needed.
License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.
Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.
Our Own Measurements
Values we measured ourselves on our server (no GPU) by actually reading and running this model’s files — not figures copied from the model card. How we measure, and the results for every model: Our Measurements.
Japanese Token Efficiency
| Tokenizer | Tokens per 1,000 Japanese characters | Ratio to the same text in English |
|---|---|---|
| This model | 546 | 0.99× |
| Qwen3 | 688 | 1.26× |
| Llama 3.2 | 744 | 1.36× |
| Gemma 3 | 564 | 1.03× |
| gpt-oss | 795 | 1.45× |
| LLM-jp-3 | 497 | 0.85× |
It needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26× as much Japanese fits in the same context length, and generation is faster per character.
Counted with the tokenizer.json of internlm/Intern-S2-397B on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.
Inside the GGUF File
File: Intern-S2-397B-Q4_K_M/Intern-S2-397B-Q4_K_M-00001-of-00007.gguf (237.10GB, Q4_K_M). We read only the header (metadata) of the file, not the weights.
| Item | Value |
|---|---|
| Architecture (as named in the GGUF) | qwen35moe |
| Maximum trained context length | 262,144 tokens |
| Layers | 61 |
| Experts | 10 active out of 512 |
| Vocabulary size | 251,392 |
| Chat template | Included (mentions tool calls, has a thinking switch) |
| imatrix | Used (Intern-S2-397B-calibration-v6.txt, 549 chunks) |
| Weight types (share of parameters) | Q4_K 76.7% / Q6_K 16.5% / Q8_0 5.1% / Q4_0 1.6% / other 0.0% |
| Average bits per weight | 5.05 bits |
| Embedding / output layer type | Q8_0 / Q8_0 |
The quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.
Recent Models in the Same Size Class
Models with over 40B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.
→ Scroll horizontally to see all columns
| Model | Parameters | Smallest VRAM tier | License | Our article |
|---|---|---|---|---|
| Qwen/Qwen3.8-Flash-Next | 180.0B | — | qwen-community-1.0 | Qwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory (2026-10-04) |
| ggml-org/GLM-5.3-Flash-GGUF | 321.3B | — | other | GLM-5.3-Flash-GGUF Vision-Language Model: ~150GB Memory (2026-10-03) |
| deepseek-ai/DeepSeek-V4.1-Flash | 763.2B | — | mit | DeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory (2026-09-10) |
How to Get It
Distributed in GGUF format, it can be obtained via the Hugging Face CLI and other tools. It is compatible with inference engines such as llama.cpp, LM Studio, koboldcpp, ramalama, Jan AI, Text Generation Web UI, LoLLMs, and Atomic Chat.
To obtain the recommended Q4_K_M quantization file and start the llama.cpp server, you can use the following commands:
pip install -U "huggingface_hub[cli]"
hf download bartowski/Intern-S2-397B-GGUF --include "Intern-S2-397B-Q4_K_M/*" --local-dir./
One-liner execution and server startup in llama.cpp:
curl -LsSf https://llama.app/install.sh | sh
llama-server -hf bartowski/Intern-S2-397B-GGUF:Q4_K_M
Note that when using speculative decoding via MTP, add the --spec-type draft-mtp startup option. When loading image inputs manually, specify the included mmproj-Intern-S2-397B-f16.gguf or similar with --mmproj.
Related Articles
- TheDrummer_Orion-26B-A4B-v1.1-GGUF Multimodal Model: 12GB+ VRAM
- vectionlabs_Salience-27B-R6-GGUF: Our Test Answers, 12GB+ VRAM
- Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: Our Test Answers
What to Read Next
- Find models by VRAM (This model does not fit even in 80GB; its smallest build needs about 102GB) → VRAM quick reference, including models over 80GB
- Engines that run this model → llama.cpp / Ollama
- What IQ1_S mean and where to get this model → GGUF format guide and models / Quantization and model-format glossary
- Other models for the same task → Other vision-language models
Sources
Update History
- 2026-09-24: Rebuilt the article: Added a “can you run it locally?" verdict from the distribution formats, converted builds, engine registries, license terms and measured compression
- 2026-09-25: Changed the title to show what the article covers (VRAM requirements, file list, etc.).
- 2026-09-28: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.