{"id":16145,"date":"2026-11-17T11:04:31","date_gmt":"2026-11-17T02:04:31","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/?p=16145"},"modified":"2026-11-17T11:13:56","modified_gmt":"2026-11-17T02:13:56","slug":"duo-1-0-real-time-audio-visual-world-models","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/duo-1-0-real-time-audio-visual-world-models\/","title":{"rendered":"Duo-1.0: Real-Time Audio-Visual Full-Duplex World Models"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/huggingface.co\/inclusionAI\/Duo-1.0-causal-base\">inclusionAI\/Duo-1.0-causal-base<\/a><\/td>\n<\/tr>\n<tr>\n<td>Publisher guide<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/publisher-inclusionai-en\/\">inclusionAI (Ant Group): models and licenses<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-11-12<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>minimax-h3-community-license<\/td>\n<\/tr>\n<tr>\n<td>Formats<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-safetensors-en\/\">safetensors<\/a><\/td>\n<\/tr>\n<tr>\n<td>Paper<\/td>\n<td><a href=\"https:\/\/arxiv.org\/abs\/2611.15131\">arXiv:2611.15131<\/a><\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>inclusionAI has released &#8220;Duo-1.0-causal-base&#8221; and &#8220;Duo-1.0-distill,&#8221; world models capable of real-time audio-visual interaction (full-duplex dialogue). These are multimodal world models that can continuously perceive incoming visual and auditory information while responding to users in real time.<\/p>\n<p>According to the paper &#8216;<a href=\"https:\/\/arxiv.org\/abs\/2611.15131\">Duo: A Reciprocal World Model for Full-Duplex Interaction<\/a>&#8216;, this model is based on the theoretical framework of the &#8220;Reciprocal World Model (RWM).&#8221; While many conventional world models focus primarily on simulating physical reality and treat reasoning and decision-making as external controls, RWM treats both intelligence and physical reality as equal components of the world. Specifically, it introduces two foundational elements: &#8220;Intent,&#8221; which handles the cognition and reasoning underlying dialogue, and &#8220;Realization,&#8221; which generates consistent audio-visual behavior based on it.<\/p>\n<p>This model constantly perceives real-time audio, video, and text inputs, enabling full-duplex dialogue that includes spontaneous conversation and interruption handling. It is designed for a wide range of interaction styles beyond screen and text inputs, including sharing physical objects, manipulating and reconstructing virtual scenes, guiding assistants, and playing audio-visual games without a mouse or keyboard.<\/p>\n<h2>Specifications<\/h2>\n<p>The following specifications have been released in the official model card and distribution package:<\/p>\n<ul>\n<li><strong>Model Version<\/strong>: Duo 1.0<\/li>\n<li><strong>Variants<\/strong>: Causal base (base model bundle), Distill (4-step distillation model)<\/li>\n<li><strong>Package Contents<\/strong>: Realization weights and backbone, Intent model and adapters, video codec, audio codec, speech recognizer, forced aligner<\/li>\n<li><strong>Realization Save Format<\/strong>: PyTorch Distributed Checkpoint (DCP)<\/li>\n<li><strong>Default Sampler<\/strong>: Euler sampler (50 steps, flow shift 1.0)<\/li>\n<\/ul>\n<h2>Performance<\/h2>\n<p>The model card and public materials do not contain comparative numerical tables for standard conventional benchmarks. Technical verification and performance characteristics based on the paper abstract and publisher&#8217;s notes are as follows:<\/p>\n<p>First, regarding dialogue control, autoregressive modeling is not limited to instructing diffusion models; it is directly utilized to control continuous dialogue reasoning and participation decisions in the conversation. On the timeline, chunk-level conditioning and token synchronization are performed, and module-specific attention masks are applied to strictly maintain causal dependencies that prevent access to future input data.<\/p>\n<p>Additionally, as a mitigation strategy for distribution shift occurring in long-term dialogue and generation, &#8220;dual rollout distillation&#8221; is applied, training using autoregressively generated Intent tokens and synthesized acoustic-visual history. For the lightweight model &#8220;Duo-1.0-distill,&#8221; 4-step distillation is applied to the Realization layer, which works in combination with the base model&#8217;s components to achieve inference efficiency and fast response performance.<\/p>\n<h2>Strengths and Use Cases<\/h2>\n<p>The Duo-1.0 model family integrates video and audio generation into a world model, specializing in real-time audio-visual interaction (full-duplex dialogue).<\/p>\n<p>The primary strength of these models is interactive tasks where they continuously perceive real-time visual and auditory inputs without interruption and respond to user inputs without latency. Function tags such as <code>video-generation<\/code>, <code>audio-generation<\/code>, and <code>world-model<\/code> are specified, assuming advanced multimodal processing that handles audio and video simultaneously.<\/p>\n<p>Specific expected use cases and application scenarios include:<\/p>\n<ul>\n<li><strong>Spontaneous Conversation and Interruption Handling<\/strong>: Smooth voice dialogue including &#8220;interruptions&#8221; where the user speaks in the middle of a response, and spontaneous speech from the system side.<\/li>\n<li><strong>Physical Object Sharing and Recognition<\/strong>: Real-time sharing of real-world objects via camera and real-time responses taking them into account.<\/li>\n<li><strong>Virtual Scene Manipulation and Reconstruction<\/strong>: Operations that dynamically change virtual scenes and environments according to user input.<\/li>\n<li><strong>Assistant Guidance and Game Operations<\/strong>: Intuitive game play and operational guides driven solely by voice or video gestures without using traditional input devices like a mouse or keyboard.<\/li>\n<\/ul>\n<p>These features make it suitable for building highly immersive interactive experiences that go beyond simple text or audio responses.<\/p>\n<h2>How to Get It<\/h2>\n<p>Duo-1.0 model weights and packages can be obtained from Hugging Face using the <code>hf<\/code> command-line tool.<\/p>\n<p>To download the base model <code>inclusionAI\/Duo-1.0-causal-base<\/code>, run the following command:<\/p>\n<pre><code class=\"language-bash\">hf download inclusionAI\/Duo-1.0-causal-base --local-dir weights\n<\/code><\/pre>\n<p>When using the distillation version <code>inclusionAI\/Duo-1.0-distill<\/code>, you must first download the complete bundle of <code>Duo-1.0-causal-base<\/code> and then overlay the weights of the distillation version into the same directory. The shared Intent model, various codecs, audio models, and Realization backbone are supplied by the base model.<\/p>\n<pre><code class=\"language-bash\">hf download inclusionAI\/Duo-1.0-causal-base --local-dir weights\nhf download inclusionAI\/Duo-1.0-distill --local-dir weights\n<\/code><\/pre>\n<p>Various configuration files, runtime integration methods, and the complete source code are summarized in the <a href=\"https:\/\/github.com\/inclusionAI\/Duo\">inclusionAI\/Duo<\/a> GitHub repository.<\/p>\n<p><!-- lmw:runnability --><\/p>\n<h2>Can You Run It Locally?<\/h2>\n<p>The publisher distributes this model as safetensors.<\/p>\n<p><strong>License \u2014 <code>minimax-h3-community-license<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\n<p><em>Compiled by this site&#8217;s code from the published formats and the license field. License summaries are not legal advice \u2014 check the publisher&#8217;s original terms before relying on them.<\/em><\/p>\n<p><!-- \/lmw:runnability --><\/p>\n<p><!-- lmw:lab --><\/p>\n<h2>Our Own Measurements<\/h2>\n<h3>Measurements We Did Not Take<\/h3>\n<p>We have not confirmed that this site meets the commercial-use terms of this model&#8217;s license (<code>minimax-h3-community-license<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation).<\/p>\n<p><!-- \/lmw:lab --><\/p>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/duo-12-causal-base-released\/\">Duo-1.2-causal-base Released for Real-time Multimodal Models<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/duo-1-0-distill-quant-fp8-realization-weights\/\">Duo-1.0-distill-quant FP8 Quantized Realization Model: 80GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/inclusionai-sing-vl-s1-4b-gguf\/\">SingVL_S1_4B_GGUF 4B Multimodal Decision-Making Model: 4GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/zoomgain-9b-gguf-released\/\">ZoomGain-9B-GGUF Vision-Language Model: 8GB+ VRAM<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Formats this model is available in<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-safetensors-en\/\">Safetensors format guide and models<\/a><\/li>\n<li><strong>Learn about the publisher<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/publisher-inclusionai-en\/\">inclusionAI (Ant Group): models, licenses and articles<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/huggingface.co\/inclusionAI\/Duo-1.0-causal-base\">https:\/\/huggingface.co\/inclusionAI\/Duo-1.0-causal-base<\/a><\/li>\n<li><a href=\"https:\/\/huggingface.co\/inclusionAI\/Duo-1.0-distill\">https:\/\/huggingface.co\/inclusionAI\/Duo-1.0-distill<\/a><\/li>\n<li><a href=\"https:\/\/arxiv.org\/abs\/2611.15131\">https:\/\/arxiv.org\/abs\/2611.15131<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/inclusionAI\/Duo\">https:\/\/github.com\/inclusionAI\/Duo<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Learn about Duo-1.0-causal-base and Duo-1.0-distill, real-time audio-visual world models for full-duplex interaction by inclusionAI.<\/p>\n","protected":false},"author":1,"featured_media":16144,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[4989,4991,4993,1691,1547,3240,4995],"class_list":["post-16145","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-audio-generation-en","tag-duo-1-0-en","tag-duo-1-0-causal-en","tag-inclusionai-en","tag-verified","tag-video-generation-en","tag-world-model-en"],"lang":"en","translations":{"en":16145,"ja":16143},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/16145","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=16145"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/16145\/revisions"}],"predecessor-version":[{"id":18114,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/16145\/revisions\/18114"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/16144"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=16145"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=16145"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=16145"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}