Nvidia published a walkthrough for running its Dynamo inference framework on Kubernetes using AI Cluster Runtime (AICR) recipes. A Dynamo deployment is declared as a DynamoGraphDeployment custom resource, which moved to API version v1beta1 in the 1.4.2 chart, but the deployment still rests on a cluster layer: a matching GPU driver stack, GPUs exposed through Kubernetes Dynamic Resource Allocation, gang scheduling for workers, node health signals before workloads land, metrics, and a gateway path. AICR pins those components in a recipe - GPU Operator, Node Feature Discovery, the DRA driver, KAI Scheduler, Grove, NVSentinel where supported, a Prometheus stack and agentgateway - and records versions, node selectors, checksums and validation checks. With Dynamo 1.4 the default path no longer uses NATS, which stays opt-in; requests travel over TCP while KV-cache events from vLLM prefix caching go over ZeroMQ, so the frontend can route a request to a worker that already holds the relevant blocks, and prefill and decode can be scaled as separate pools.
Our takeTreating KV-cache locality as a scheduling problem is the real change here: routing to the worker that already holds the prefix avoids reprocessing, but that only works if placement respects topology and the cache-event plane is fast enough to trust - hence ZeroMQ on the hot path instead of a message broker, at the cost of one more component to secure. The reproducible part is the pinning: checksums, node selectors and validation checks turn 'it worked on my cluster' into something another team can falsify.