NVIDIA-Nemotron-Labs-Dory1.0-4B-GGUF Text Generation Model: 8GB+ VRAM

NVIDIA-Nemotron-Labs-Dory1.0-4B-GGUF Text Generation Model: 8GB+ VRAM

At a Glance

Item Value
Repository nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-GGUF
Family guide NVIDIA-Nemotron-Labs-Dory1.0 guide (2 articles)
Publisher guide NVIDIA: models and licenses
Published 2026-10-17
License openmdw-1.1
Formats GGUF
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

NVIDIA has released “NVIDIA-Nemotron-Labs-Dory1.0-4B-GGUF", a 4-billion parameter language model that adopts a potential recurrent loop structure.

This model employs a normalized-GPT (normalized-GPT) architecture consisting of an input block (8 layers), a recurrent block (20 layers), and an output block (8 layers). A major feature is that by repeatedly applying the same recurrent block, it can dynamically change the inference computation per token and adjust accuracy without increasing the number of parameters.

It is designed to solve problems with fewer output token counts by deepening computation (thinking) within the latent space, and is available for both commercial and non-commercial use.

Specifications

  • Parameters: 4B
  • Architecture: Looped normalized-GPT with alternating sliding window attention (SWA) and full attention. It consists of an input block of 8 layers, a shared recurrent block of 20 layers, and an output block of 8 layers
  • Context length: Up to 256K tokens

Performance

Below are the benchmark evaluation results listed in the model card. All measurements are scores conducted by NVIDIA in the same evaluation environment (NeMo Gym / Nemo Evaluator SDK, Terminal-Bench uses the Terminus-2 harness).

→ Scroll horizontally to see all columns

Task Dory1.0-4B / 1 loop Dory1.0-4B / 2 loops Dory1.0-4B / 4 loops Qwen3.5-4B Qwen3.5-9B Gemma 4 26B A4B Qwen3.6-35B-A3B
Coding & Agentic
SciCode 27.3 34.94 35.11 25.82 28.21 40.32 37.24
SWE-bench Verified 50.3 59.5 61.8 54.7 61.0 57.40 73.4†
Terminal-Bench 2.1 12.5 18.1 22.2 15 24 35 43
τ³-bench (Banking) 11.6 12.89 14.69 5.03 7.22 13.66 9.54
GDPval-AA-V2 (Elo) 815 946 967 424 546 751.6 987
Reasoning
GPQA Diamond (no tools) 69.19 72.41 72.16 77.71 80.93 80.43 85.35
HLE (text-only, no tools) 8.25 10.89 10.43 11.31 15.43 17.79 19.18
Instruction Following
IFBench 74.29 77.94 79 53.83 59 77.89 57.44
Long Context
AA-LCR 63 71 73 64.7 68.4 64.6 68.5
General Knowledge
AA-Omniscience (accuracy) 10.03 10.08 10.03 15.02 18.05 21.13 19.95

According to the publisher’s measurement results, it can be confirmed that the model’s inference and coding performance improve step-by-step as the number of loops is increased. In particular, on SWE-bench Verified, which measures the ability to fix actual GitHub issues, it reaches 59.5% with 2 loops and 61.8% with 4 loops. Not only does this surpass the comparably sized Qwen3.5-4B (54.7%), but it also records figures exceeding larger parameter models such as Qwen3.5-9B (61.0%) and Gemma 4 26B A4B (57.40%). Furthermore, in instruction-following on IFBench, it records 79.00 at 4 loops, which is the highest value among the compared models.

On the other hand, on GPQA Diamond, which handles PhD-level scientific problems, it stops at 72.16 even at 4 loops, falling short of Qwen3.5-4B’s 77.71. Also in HLE, a set of extremely difficult expert questions, 10.89 at 2 loops is the highest value, failing to reach Qwen3.5-4B’s 11.31 or Qwen3.5-9B’s 15.43. In addition, AA-Omniscience, which measures general knowledge, shows modest results at about 10% compared to other models, showing a clear disadvantage against other models in a wide range of knowledge questions and difficult reasoning tasks.

Strengths and Use Cases

This model excels at logical processing such as agent tasks, coding, and complex instruction following. Because it features a latent recurrent loop mechanism that repeatedly executes the same weights, its strength lies in its ability to derive solutions by increasing computational volume within the latent space without outputting lengthy thinking text externally. According to the model card, increasing the number of loops allows the model to solve the same tasks more efficiently with fewer generated tokens.

In the post-training phase, multi-environment reinforcement learning via GRPO (Group Relative Policy Optimization) using NeMo RL and NeMo Gym has been applied. As a result, in addition to math, code, science, and instruction following, optimizations have been made for multi-step tool use, multi-turn conversations, and structured output environments. In addition to English, it supports 43 programming languages and can handle long contexts up to 256K tokens, making it suitable for analyzing long codebases and embedding autonomous agents.

In addition, because the number of loops during inference can be changed while keeping a single checkpoint, settings can be flexibly switched according to the use case, ranging from lightweight processing where computational cost is minimized to high-difficulty agent tasks where computational resources are invested to increase accuracy.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch)

Your VRAM Quantization File size Est. memory needed
8GB (RTX 4060 / 3060 Ti, etc.) Q5_K_M 3.4GB 4.1GB

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

Runs in Ollama, LM Studio and llama.cpp as-is.

It is distributed in GGUF, so no conversion is needed.

License — openmdw-1.1: A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.

Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Our Own Measurements

Measurements We Did Not Take

We have not confirmed that this site meets the commercial-use terms of this model’s license (openmdw-1.1). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation).

How to Get It

This model is distributed in GGUF format (Q5_K_M quantized version) and can be used with llama.cpp. The OpenMDW License Agreement, version 1.1 applies to the license, and it can be used for both commercial and non-commercial purposes in accordance with the terms of use. Since no access restrictions (gating) are set on the repository, anyone can download it directly.

To run this model at this time, it is necessary to build from dedicated llama.cpp source code that includes support for the Dory architecture. An example build on a macOS (Apple Silicon) environment is as follows:

brew install cmake openssl@3

git init llama.cpp-dory
cd llama.cpp-dory
git remote add origin https://github.com/brianchmielNV/llama.cpp.git
git fetch --depth 1 origin 98a74c97ea09ac72f6ceda1559808fc0a2f447ce
git checkout --detach FETCH_HEAD

HF_UI_VERSION=b11551 cmake -S. -B build \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_OSX_ARCHITECTURES=arm64 \
    -DCMAKE_OSX_SYSROOT="/Library/Developer/CommandLineTools/SDKs/MacOSX26.sdk" \
    -DGGML_METAL=ON -DGGML_METAL_EMBED_LIBRARY=ON \
    -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=Apple \
    -DLLAMA_BUILD_SERVER=ON \
    -DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_EXAMPLES=OFF -DLLAMA_BUILD_APP=OFF \
    -DLLAMA_BUILD_UI=OFF -DLLAMA_USE_PREBUILT_UI=ON \
    -DLLAMA_OPENSSL=ON -DOPENSSL_ROOT_DIR="$(brew --prefix openssl@3)"
cmake --build build --parallel 4 --target llama-server

After the build is complete, place the downloaded GGUF file (dory-q5_k_m.gguf) in the directory and start the server as follows:

./build/bin/llama-server \
    -m./dory-q5_k_m.gguf \
    --alias dory \
    --host 127.0.0.1 --port 8080 \
    -ngl 999 \
    --ctx-size 4096 --parallel 1 \
    --batch-size 512 --ubatch-size 512 \
    --cache-type-k f16 --cache-type-v f16 \
    --flash-attn off \
    --no-context-shift \
    --jinja --reasoning on \
    --temp 1.0 --top-p 0.95 --top-k 0 --min-p 0 --repeat-penalty 1 \
    --override-kv "dory.num_loops=int:2,dory.recurrent_kv_cache_mode=str:per_loop"

By setting dory.num_loops with the startup option --override-kv, you can adjust the number of loops during inference from 1 to 4. The standard recommended value is 2 loops, which offers a good balance between accuracy and computational volume. Also, if you want to reduce KV cache consumption per loop, you can use the last-loop KV cache mode, which overwrites and saves to a cache area equivalent to a single loop, by specifying dory.recurrent_kv_cache_mode=str:last_loop.

Related Articles

What to Read Next

Sources