NVIDIA Snapshot Cuts GPU Workload Startup to Seconds
Public- 11 Nov, 2026

NVIDIA has introduced Snapshot, an open-source component of the NVIDIA DSX AI Factory Platform that makes initialized GPU workloads reusable across inference replicas. The problem it targets is familiar to anyone running LLM serving: when demand spikes or a replica fails, available GPUs do not immediately become serving capacity, because initializing an LLM workload - a cold start - can take minutes. Snapshot is a checkpoint and restore system for GPU workloads on Kubernetes. It captures a fully initialized workload, including CPU and GPU memory, and restores that state on compatible infrastructure, bringing inference capacity online in seconds rather than minutes. Infrastructure provisioning and container image loading remain outside the measured restore path.
The tool exposes two operations for GPU Pods, both driven by Kubernetes custom resources. To checkpoint a running Pod, you create a PodSnapshot that names it; to start a Pod from a template and checkpoint it as soon as it is ready, you create a SnapshotJob instead. Restoring means adding the annotation nvidia.com/restore-from to a new Pod and naming the PodSnapshot to restore from. The workload itself only needs a small handshake to pause before the checkpoint and resume after the restore, and the repository ships ready-made examples for vLLM, SGLang, and NVIDIA TensorRT LLM. Snapshot can checkpoint and restore compatible workloads spanning multiple GPUs on a single node as a single artifact across ranks, which supports fast startup for large mixture-of-experts models using tensor parallelism, data and expert parallelism, and decode context parallelism, provided the workload meets the checkpoint/restore and runtime compatibility requirements.

Benchmarks cover models from 600 million parameters up to MoE models with 1 TB checkpoints on 8 GPUs, tested on the same hardware and software stack under two strategies: cold start versus Snapshot restore. Across all tested configurations, Snapshot brings a replica back in under 45 seconds, 8x to 36x faster than a cold start. Per-engine restore ranges are as follows: vLLM at 4.7 to 39.6 seconds, SGLang at 3.6 to 42.7 seconds, and TensorRT LLM at 7.79 to 33.55 seconds. Both measurements exclude pulling the container image and setting up the container runtime, and the KV cache release is triggered before every checkpoint, so it is not part of the artifact. Checkpoint time and artifact size are reported as a one-time cost per model and configuration.

The restore path itself breaks down into agent setup, CRIU restore, CUDA restore, and wake/remap, and the cold-start breakdown shows which initialization phases a restore skips. Snapshot was originally developed for NVIDIA Dynamo inference workers, but it is designed for integration by other inference engines, schedulers, controllers, and platforms. Benchmarking tools and reproduction instructions live in the repository, alongside installation instructions and usage guides, and the Dynamo user guide covers restoring workers. Design partners have also evaluated Snapshot on their own workloads.


