Understand this story
Story summary
The English summary is being updated. See the reporting timeline below.
Latest development · Report
Amazon SageMaker HyperPod Introduces Workload Recovery and Expanded Infrastructure ChecksDevelopments
1 confirmed developments
- Source:AWS Machine Learning
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 19
AWS Machine LearningFirst-party
Amazon SageMaker HyperPod Introduces Workload Recovery and Expanded Infrastructure Checks
Amazon SageMaker HyperPod has introduced workload recovery and infrastructure checks designed to detect distributed training failures earlier and increase the proportion of time spent on productive training. The new capabilities include MultiNodeTrainingValidation for EKS clusters, the Kubernetes Resource Monitoring Agent, and hung job detection for Slurm clusters. Hung job detection can log CloudWatch events, cancel jobs according to predefined rules, and then hand them back to Slurm for requeuing.
AI Trends