NVIDIA AICR validates Kubernetes support for Vera Rubin NVL72

  • PublicPublic
  • 03 Nov, 2026
NVIDIA AICR validates Kubernetes support for Vera Rubin NVL72

Configuring Kubernetes for new GPU hardware like NVIDIA Vera Rubin is complex and error-prone, requiring precise alignment of drivers, kernels, runtimes, and schedulers. NVIDIA AI Cluster Runtime (AICR), part of the DSX AI Factory Platform, simplifies this by capturing tuned configurations in reusable recipes. It generates bundles for deployment tools and validates running clusters, ensuring environments are ready for Vera Rubin NVL72 compute trays on Day 1. This approach helps platform teams design, build, and operate efficient AI factories with open source, modular software components.

Figure 1. AICR validation on Vera Rubin produces signed, publicly verifiable evidence and feeds results back to the upstream components.
Figure 1. AICR validation on Vera Rubin produces signed, publicly verifiable evidence and feeds results back to the upstream components.

For teams currently running GB200 NVL72, Vera Rubin NVL72 is the third generation of the MGX rack-scale architecture, with several user-visible changes. To validate support end-to-end, the AICR runtime bundle was deployed on Vera Rubin clusters using GPU Operator v26.7.0, DRA driver v0.5.0, and a host-installed R615 driver. Both training and inference recipe variants passed end-to-end validation. The training variant includes a performance gate: a cross-node NCCL all-reduce over the NVLink fabric via an IMEX ComputeDomain, gated on a bus-bandwidth floor. Early validation faced a temporary hardware/fabric issue in one cluster, but reseating the hardware resolved it, allowing all GPUs to join a single fabric clique.

AICR captures platform-specific decisions as a versioned recipe, making configurations explicit and reproducible. The new Vera Rubin recipe family (service: rke2, accelerator: vr200, OS: Ubuntu) ships as a Preview, without full production support while the platform is pre-release. The first deployment targeted a novel combination: arm64, Ubuntu 26.04, a 64k-page kernel, RKE2, and Vera Rubin, which produced a healthy runtime within hours. A second cluster reproduced this result the next day, demonstrating the recipe's repeatability. This versioned approach ensures that the specific decisions made during bring-up are preserved and can be verified across different clusters.

The AICR workflow, executed with the v1.0.0 toolchain, begins with offline recipe generation. The command aicr recipe resolves criteria against the catalog, producing a YAML file that layers five overlays into 14 components. Key components include the GPU Operator, DRA driver, DraNet for ConnectX-9 device allocation, and NodeWright for exclusive RDMA netns mode. The bundle generation command aicr bundle creates a set of ordered Helm charts and a deploy script. This bundle includes NodeWright, KAI Scheduler, NVSentinel, NFD, and the Prometheus stack, with all component versions pinned to the configuration validated on Vera Rubin hardware.

Validation involves capturing the cluster state and checking readiness constraints. The command aicr snapshot saves the state, while aicr validate runs the deployment, conformance, and performance phases. The deployment phase checks the OS version and kernel details, while the conformance phase verifies component health. The performance phase runs an NCCL all-reduce as an in-cluster job, checking bus bandwidth against the floor defined in the recipe. The validator provisions the ComputeDomain and IMEX ResourceClaimTemplate itself, requiring no operator prerequisites. The inference variant, which uses the Dynamo platform, passes the same deployment and conformance checks on the identical cluster.

A critical end-to-end test involves a two-node ComputeDomain where IMEX channels are claimed by pods on each compute tray. For host-managed IMEX images, the host's static nvidia-imex service must be disabled and masked, as it is mutually exclusive with DRA-managed ComputeDomains. GPU Operator v26.7.0+ ships a stricter CRD requiring that numNodesbe explicitly set, which affects fresh cluster installs but not upgrades. Applying the test manifest results in two running pods, each with an IMEX channel injected, taking about 12 seconds. The entire sequence from recipe generation to validation completes in roughly five minutes, with the performance phase adding the runtime of the NCCL job.