Duo-1.0: Real-Time Audio-Visual Full-Duplex World Models

Duo-1.0: Real-Time Audio-Visual Full-Duplex World Models

At a Glance

Item Value
Repository inclusionAI/Duo-1.0-causal-base
Publisher guide inclusionAI (Ant Group): models and licenses
Published 2026-11-12
License minimax-h3-community-license
Formats safetensors
Paper arXiv:2611.15131
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

inclusionAI has released “Duo-1.0-causal-base" and “Duo-1.0-distill," world models capable of real-time audio-visual interaction (full-duplex dialogue). These are multimodal world models that can continuously perceive incoming visual and auditory information while responding to users in real time.

According to the paper 'Duo: A Reciprocal World Model for Full-Duplex Interaction', this model is based on the theoretical framework of the “Reciprocal World Model (RWM)." While many conventional world models focus primarily on simulating physical reality and treat reasoning and decision-making as external controls, RWM treats both intelligence and physical reality as equal components of the world. Specifically, it introduces two foundational elements: “Intent," which handles the cognition and reasoning underlying dialogue, and “Realization," which generates consistent audio-visual behavior based on it.

This model constantly perceives real-time audio, video, and text inputs, enabling full-duplex dialogue that includes spontaneous conversation and interruption handling. It is designed for a wide range of interaction styles beyond screen and text inputs, including sharing physical objects, manipulating and reconstructing virtual scenes, guiding assistants, and playing audio-visual games without a mouse or keyboard.

Specifications

The following specifications have been released in the official model card and distribution package:

  • Model Version: Duo 1.0
  • Variants: Causal base (base model bundle), Distill (4-step distillation model)
  • Package Contents: Realization weights and backbone, Intent model and adapters, video codec, audio codec, speech recognizer, forced aligner
  • Realization Save Format: PyTorch Distributed Checkpoint (DCP)
  • Default Sampler: Euler sampler (50 steps, flow shift 1.0)

Performance

The model card and public materials do not contain comparative numerical tables for standard conventional benchmarks. Technical verification and performance characteristics based on the paper abstract and publisher’s notes are as follows:

First, regarding dialogue control, autoregressive modeling is not limited to instructing diffusion models; it is directly utilized to control continuous dialogue reasoning and participation decisions in the conversation. On the timeline, chunk-level conditioning and token synchronization are performed, and module-specific attention masks are applied to strictly maintain causal dependencies that prevent access to future input data.

Additionally, as a mitigation strategy for distribution shift occurring in long-term dialogue and generation, “dual rollout distillation" is applied, training using autoregressively generated Intent tokens and synthesized acoustic-visual history. For the lightweight model “Duo-1.0-distill," 4-step distillation is applied to the Realization layer, which works in combination with the base model’s components to achieve inference efficiency and fast response performance.

Strengths and Use Cases

The Duo-1.0 model family integrates video and audio generation into a world model, specializing in real-time audio-visual interaction (full-duplex dialogue).

The primary strength of these models is interactive tasks where they continuously perceive real-time visual and auditory inputs without interruption and respond to user inputs without latency. Function tags such as video-generation, audio-generation, and world-model are specified, assuming advanced multimodal processing that handles audio and video simultaneously.

Specific expected use cases and application scenarios include:

  • Spontaneous Conversation and Interruption Handling: Smooth voice dialogue including “interruptions" where the user speaks in the middle of a response, and spontaneous speech from the system side.
  • Physical Object Sharing and Recognition: Real-time sharing of real-world objects via camera and real-time responses taking them into account.
  • Virtual Scene Manipulation and Reconstruction: Operations that dynamically change virtual scenes and environments according to user input.
  • Assistant Guidance and Game Operations: Intuitive game play and operational guides driven solely by voice or video gestures without using traditional input devices like a mouse or keyboard.

These features make it suitable for building highly immersive interactive experiences that go beyond simple text or audio responses.

How to Get It

Duo-1.0 model weights and packages can be obtained from Hugging Face using the hf command-line tool.

To download the base model inclusionAI/Duo-1.0-causal-base, run the following command:

hf download inclusionAI/Duo-1.0-causal-base --local-dir weights

When using the distillation version inclusionAI/Duo-1.0-distill, you must first download the complete bundle of Duo-1.0-causal-base and then overlay the weights of the distillation version into the same directory. The shared Intent model, various codecs, audio models, and Realization backbone are supplied by the base model.

hf download inclusionAI/Duo-1.0-causal-base --local-dir weights
hf download inclusionAI/Duo-1.0-distill --local-dir weights

Various configuration files, runtime integration methods, and the complete source code are summarized in the inclusionAI/Duo GitHub repository.

Can You Run It Locally?

The publisher distributes this model as safetensors.

License — minimax-h3-community-license: A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.

Compiled by this site’s code from the published formats and the license field. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Our Own Measurements

Measurements We Did Not Take

We have not confirmed that this site meets the commercial-use terms of this model’s license (minimax-h3-community-license). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation).

Related Articles

What to Read Next

Sources