{"id":14443,"date":"2026-11-17T11:03:10","date_gmt":"2026-11-17T02:03:10","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/?p=14443"},"modified":"2026-11-17T11:12:33","modified_gmt":"2026-11-17T02:12:33","slug":"nvidia-aicr-validated-kubernetes-dynamo-runtimes","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/nvidia-aicr-validated-kubernetes-dynamo-runtimes\/","title":{"rendered":"NVIDIA Releases AICR for Validated Kubernetes Dynamo Runtimes"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Publisher guide<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/publisher-nvidia-en\/\">NVIDIA: models and licenses<\/a><\/td>\n<\/tr>\n<tr>\n<td>Publisher<\/td>\n<td>NVIDIA Developer<\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-10-31<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>NOASSERTION<\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>NVIDIA has published a guide on how to build validated runtimes using NVIDIA AI Cluster Runtime (AICR), a tool designed to ensure the stable operation of the distributed inference framework NVIDIA Dynamo on Kubernetes. AICR packages runtime requirements\u2014such as GPU driver stacks, Dynamic Resource Allocation (DRA), gang scheduling, node health signals, and Prometheus metrics\u2014into reproducible &#8220;recipes&#8221; with fixed component versions.<\/p>\n<p>The most significant change in this release is that Dynamo 1.4.2 now standardizes on a configuration using TCP for the request plane and ZMQ for KV-cache events. This enables cache-aware routing for both the native Dynamo Frontend router and paths utilizing the Gateway API Inference Extension (GAIE) and Endpoint Picker Plugin (EPP).<\/p>\n<h2>Breaking Changes and Deprecations<\/h2>\n<p>Starting with Dynamo 1.4, frontends, workers, and optional GAIE\/EPP components discover each other through Kubernetes resources. Consequently, the default AICR path no longer uses NATS, and NATS installation has also been removed from the default settings of the <code>dynamo-platform<\/code> chart.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th style=\"text-align: left;\">Item<\/th>\n<th style=\"text-align: left;\">Old Setting\/Specification<\/th>\n<th style=\"text-align: left;\">New Setting\/Specification<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td style=\"text-align: left;\">NATS Usage<\/td>\n<td style=\"text-align: left;\">Installed by default<\/td>\n<td style=\"text-align: left;\">Available as an option (opt-in) only<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\">API Version<\/td>\n<td style=\"text-align: left;\">v1alpha1<\/td>\n<td style=\"text-align: left;\">v1beta1 (DynamoGraphDeployment CRD)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>This change impacts platform teams planning to operate Dynamo with a Kubernetes-native configuration. Specific deployments that require NATS must enable it explicitly.<\/p>\n<h2>Key Changes<\/h2>\n<h3>Introduction of Two Routing Paths<\/h3>\n<p>Two routing paths are now available for operating Dynamo on Kubernetes depending on the use case. One path uses the &#8220;Dynamo Frontend router,&#8221; which is suitable for minimal configurations with model-specific endpoints. The other uses &#8220;Gateway API \/ EPP,&#8221; which is suitable when integrating with shared Kubernetes ingresses managed through Gateway API resources. Both paths utilize KV-cache events published by workers to achieve accurate cache-aware routing.<\/p>\n<h3>Introduction of Validated Recipes via AICR<\/h3>\n<p>AICR reduces the effort required to manage infrastructure dependencies. It supports reproducible environment setup through a five-step process: capturing a cluster snapshot, generating a Dynamo inference recipe, rendering a deployment bundle, deploying via Helmfile or GitOps, and performing post-installation runtime verification. This allows platform teams to automatically verify that complex components\u2014such as the GPU Operator, KAI Scheduler, and Prometheus stack\u2014are correctly configured for a specific Dynamo version.<\/p>\n<h3>Changes to Communication Protocols for KV-cache Events<\/h3>\n<p>Dynamo 1.4.2 uses TCP for request forwarding and ZMQ for transmitting KV-cache events. For example, when using vLLM workers, you must configure KV-cache events to be emitted via the ZMQ publisher using settings such as the following:<\/p>\n<pre><code class=\"language-bash\">--enable-prefix-caching --kv-transfer-config \\\n{&quot;enable_kv_cache_events&quot;: true, &quot;publisher&quot;: &quot;zmq&quot;, &quot;endpoint&quot;: &quot;tcp:\/\/*:5557&quot;}\n<\/code><\/pre>\n<p>With this configuration, the Dynamo worker runtime subscribes to local vLLM event sources and republishes normalized events to the Dynamo event plane, enabling the router to make decisions based on the latest cache state.<\/p>\n<h2>Supported Models and Hardware<\/h2>\n<p>AICR&#8217;s Dynamo inference recipes fix component versions to match specific hardware environments. For instance, the AICR v1.0.0 recipe for NVIDIA GB200 NVL72 and NVIDIA GB300 NVL72 systems in an Amazon EKS environment selects the following validated components:<\/p>\n<ul>\n<li>NVIDIA GPU Operator v26.7.1<\/li>\n<li>NVIDIA DRA GPU Driver 0.5.0<\/li>\n<li>KAI Scheduler v0.16.9<\/li>\n<li>Grove v0.1.0-alpha.13<\/li>\n<li>Dynamo Platform 1.4.2<\/li>\n<\/ul>\n<p>Additionally, Dynamo supports orchestration for running inference engines such as vLLM, SGLang, and NVIDIA TensorRT LLM within pods on Kubernetes.<\/p>\n<h2>How to Update<\/h2>\n<p>The following procedure is recommended for deploying runtimes using AICR. First, capture a cluster snapshot, and then generate the inference recipe:<\/p>\n<pre><code class=\"language-bash\"># Capture cluster snapshot\naicr snapshot --namespace aicr-validation --output snapshot.yaml\n\n# Generate Dynamo inference recipe\naicr recipe --snapshot snapshot.yaml --intent inference --platform dynamo --output recipe.yaml\n<\/code><\/pre>\n<p>Next, render the deployment bundle from the recipe. If deploying locally using Helmfile, use the following commands:<\/p>\n<pre><code class=\"language-bash\"># Render bundle\naicr bundle --recipe recipe.yaml [other flags] --deployer helmfile --output bundle\n\n# Execute deployment\nhelmfile -f bundle\/helmfile.yaml diff\nhelmfile -f bundle\/helmfile.yaml apply\n<\/code><\/pre>\n<p>This material also describes production deployment methods using GitOps (ArgoCD or Flux).<\/p>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/nvidia-snapshot-kubernetes-gpu-checkpoint-restore\/\">NVIDIA Snapshot: Accelerate GPU Workload Cold Starts on Kubernetes<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/17\/tensorrt-edge-llm-mlperf-edge-agentic-jetson-thor\/\">TensorRT Edge-LLM Accelerates MLPerf Agentic Benchmark<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Engines mentioned in this article<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-vllm-en\/\">vLLM<\/a><\/li>\n<li><strong>Learn about the publisher<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/publisher-nvidia-en\/\">NVIDIA: models, licenses and articles<\/a><\/li>\n<li><strong>Models NVIDIA has released<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/nvidia-metrictrail-release\/\">MetricTrail: File List<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/nvidia-cosmos3-h-dreams-surgical-world-model\/\">Cosmos3-H-Dreams Video Generation Model: 48GB+ VRAM, File List<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/nvidia-nemotron-labs-dory1-4b-gguf\/\">NVIDIA-Nemotron-Labs-Dory1.0-4B-GGUF Text Generation Model: 8GB+ VRAM<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/developer.nvidia.com\/blog\/build-a-validated-kubernetes-runtime-for-nvidia-dynamo\/\">Build a Validated Kubernetes Runtime for NVIDIA Dynamo<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Learn how NVIDIA AI Cluster Runtime (AICR) helps build validated, reproducible runtimes for NVIDIA Dynamo on Kubernetes.<\/p>\n","protected":false},"author":1,"featured_media":14442,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[105],"tags":[4469,2023,4471,1547,592,4473],"class_list":["post-14443","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-engines-and-tools","tag-aicr-en","tag-kubernetes-en","tag-nvidia-dynamo-en","tag-verified","tag-vllm-en","tag-zmq-en"],"lang":"en","translations":{"en":14443,"ja":14441},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/14443","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=14443"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/14443\/revisions"}],"predecessor-version":[{"id":18014,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/14443\/revisions\/18014"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/14442"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=14443"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=14443"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=14443"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}