NVIDIA Dynamo Details EPD Disaggregation for Multimodal
Overview
On September 9, 2026, NVIDIA Developer published a blog post detailing how to utilize “Encode-Prefill-Decode (EPD) disaggregation" to accelerate multimodal model serving. By using the NVIDIA Dynamo framework to separate vision encoder processing from the LLM prefill and decode stages, it is reported to achieve up to a 5x reduction in Time to First Token (TTFT) and up to a 7x reduction in end-to-end response time for image-heavy prompts.
Announcement Details
In multimodal model inference, media preprocessing and running Vision Transformers (ViT) to generate embeddings are required before LLM prefill begins. In traditional aggregated serving, these share a single worker and scheduling domain, causing delays when media volume increases. NVIDIA Dynamo implements Encode-Prefill-Decode (EPD) disaggregation, separating the roles of encoders and Prefill-Decode (PD) workers.
The documentation presents three topologies for encoder deployment:
– Aggregated: Each GPU runs a single aggregated worker, and the scheduler manages encoding, prefill, and decode within the same request lifecycle.
– Colocated encoder: One or more encoder workers are colocated with PD workers on each GPU, sharing GPU computing while maintaining separate request queues and batching. This is considered suitable for homogeneous clusters.
– Disaggregated encoder: The primary GPU tier hosts PD workers while encoder workers run on a lower-cost GPU tier suitable for encoding tasks. Vision embeddings are transferred to the PD tier via the NVIDIA Inference Transfer Library (NIXL).
Additionally, EPD is most effective for prompts with many images, short-to-medium outputs, and quantized Mixture-of-Experts (MoE) models. While it achieves up to a 5x reduction in TTFT and a 7x reduction in end-to-end response time compared to aggregated serving in image-centric prompts, the benefits are reported to diminish when decoding accounts for the majority of latency or when large dense models reduce the computational share of the vision encoder.
In mixed text and multimodal traffic environments, eliminating head-of-line blocking reduces average TTFT for text requests by 42.2% and for image requests by 30.8%. Furthermore, when LLM weights are quantized to NVFP4 while keeping the vision encoder in BF16, the goodput of colocated EPD improves from 1.78x to 2.64x compared to aggregated serving.
Impact on Local LLM Users
For engineers running open-weight models locally or serving them in distributed environments, this expands the options for inference optimization using NVIDIA Dynamo. NVIDIA Dynamo is provided as an open-source inference framework, and the documentation notes that experiments can be reproduced through the GitHub guide at ai-dynamo/dynamo.
The material outlines specific implementation approaches such as separating encoder and PD workers, leveraging the NVIDIA Inference Transfer Library (NIXL), enabling parallel media decoding, embedding caching, and multimodal KV routing, providing guidelines for selecting the optimal topology according to hardware configuration and traffic characteristics. The documentation does not contain direct mentions of existing model license formats or future open-weight release policies.