AI News

Trending story

NVIDIA open-sources Snapshot, a snapshot and restore system for GPU workloads

1 reports1 story updatesUpdated Nov 10 23:00

Understand this story

Story summary

The English summary is being updated. See the reporting timeline below.

Latest development · Report

NVIDIA Open-Sources Snapshot, Cutting GPU Workload Recovery to 3.6 to 42.7 Seconds

Developments

1 confirmed developments

  1. NVIDIA open-sources GPU workload snapshot and restore system Snapshot

    Source:NVIDIA Developer · Generative AI

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 10

  1. NVIDIA Developer · Generative AIFirst-partySelected

    NVIDIA Open-Sources Snapshot, Cutting GPU Workload Recovery to 3.6 to 42.7 Seconds

    NVIDIA has released an open-source component, NVIDIA Snapshot, which can checkpoint and restore GPU workloads on Kubernetes. In testing, restoring inference replicas took 3.6 to 42.7 seconds, an 8x to 36x speedup compared with cold starts that take several minutes. The component supports tensor parallelism, data and expert parallelism, and decode context parallelism, and can handle MoE models with approximately 1 TB checkpoints across 8 GPUs, with checkpoint creation taking 15 seconds to 5.5 minutes. The repository already provides examples for vLLM, SGLang, and TensorRT LLM, and restoration requires the target nodes to use the same GPU model and driver.

Related stories

  1. Related storiesAWS open-sources the agent evaluation framework UAEF
  2. Related storiesHao AI Lab integrates MiniMax H3 Flex into FastVideo
  3. Related storiesHao AI Lab announces that FastVideo supports Open-H3-Regenerate-2K
  4. Related storiesHao AI Lab open-sources the JetSpec++ model, code, and inference engine