Understand this story
The English summary is being updated. See the reporting timeline below.
Latest development · Report
NVIDIA Open-Sources Snapshot, Cutting GPU Workload Recovery to 3.6 to 42.7 SecondsDevelopments
1 confirmed developments
NVIDIA open-sources GPU workload snapshot and restore system Snapshot
Source:NVIDIA Developer · Generative AI
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 10
NVIDIA Developer · Generative AIFirst-partySelected
NVIDIA Open-Sources Snapshot, Cutting GPU Workload Recovery to 3.6 to 42.7 Seconds
NVIDIA has released an open-source component, NVIDIA Snapshot, which can checkpoint and restore GPU workloads on Kubernetes. In testing, restoring inference replicas took 3.6 to 42.7 seconds, an 8x to 36x speedup compared with cold starts that take several minutes. The component supports tensor parallelism, data and expert parallelism, and decode context parallelism, and can handle MoE models with approximately 1 TB checkpoints across 8 GPUs, with checkpoint creation taking 15 seconds to 5.5 minutes. The repository already provides examples for vLLM, SGLang, and TensorRT LLM, and restoration requires the target nodes to use the same GPU model and driver.