Understand this story
Amazon SageMaker HyperPod has launched Global Inference Gateway, which routes inference requests among multiple clusters in the same region or across regions through a single managed endpoint. The gateway selects clusters based on available KV cache capacity, queue depth, regional proximity, and error rates. It also supports automatic failover, SigV4 and IAM authentication, and private connectivity through AWS PrivateLink. The feature is currently available in 13 AWS Regions, with no other evidence-backed developments at this time.
AI-generated from public reports
Latest development · Report
Amazon SageMaker HyperPod Launches Cross-Region Global Inference GatewayDevelopments
1 confirmed developments
- Source:AWS Machine Learning
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 21
AWS Machine LearningFirst-party
Amazon SageMaker HyperPod Launches Cross-Region Global Inference Gateway
Amazon SageMaker HyperPod has launched Global Inference Gateway, which routes inference requests through a single managed endpoint across multiple clusters in the same region or across regions. The gateway selects clusters based on available KV cache capacity, queue depth, regional proximity, and error rates, and supports automatic failover, SigV4 and IAM authentication, and private connections through AWS PrivateLink. The feature is now available in 13 AWS regions.
AI Trends