Understand this story
On November 17, 2026, Amazon Web Services announced that Amazon SageMaker HyperPod Inference supports managed NVIDIA Dynamo, combining Dynamo's distributed inference capabilities with HyperPod's infrastructure management. AWS Machine Learning explained that the support covers multi-node deployment, prefill/decode disaggregation, and KV-aware routing; taking GLM-5.3 as an example, its FP8 weights exceed the 640 GB of GPU memory on a single ml.p5.48xlarge, requiring cross-node deployment. Deployment can optionally use dynamoSpec with a bring-your-own DGD or InferenceEndpointConfig, and the inference engine supports vLLM, SGLang, and TensorRT-LLM, with vLLM as the default. Unite.AI reported the same day that in a Qwen3-235B-A22B-FP8 test, disaggregated deployment reached 207 output tokens per second per GPU at a concurrency of 80, higher than the 101 achieved by aggregated deployment at a concurrency of 32. The support is currently in a generally available state following the announcement.
AI-generated from public reports
Latest development · Report
AWS Hosts NVIDIA Dynamo in SageMaker HyperPod InferenceDevelopments
1 confirmed developments
AWS adds managed NVIDIA Dynamo support to SageMaker HyperPod Inference
Source:AWS Machine Learning
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 18
Unite.AI
AWS Hosts NVIDIA Dynamo in SageMaker HyperPod Inference
On November 17, 2026, AWS announced that SageMaker HyperPod Inference supports NVIDIA Dynamo for hosted multi-node and disaggregated model serving. In tests with Qwen3-235B-A22B-FP8, the disaggregated deployment reached 207 output tokens/second per GPU at concurrency 80, higher than the 101 achieved by the aggregated deployment at concurrency 32. HyperPod provides EFA networking, GPU scheduling, health monitoring, and node replacement, and users can take the managed path via the dynamoSpec of InferenceEndpointConfig.
AWS Machine LearningFirst-party
Amazon SageMaker HyperPod Inference now hosts NVIDIA Dynamo
Amazon SageMaker HyperPod Inference now supports hosting NVIDIA Dynamo, combining Dynamo's distributed inference capabilities with HyperPod's infrastructure management, and supporting multi-node deployment, prefill/decode disaggregation, and KV-aware routing. Taking GLM-5.3 as an example, its FP8 weights exceed the 640 GB GPU memory of a single ml.p5.48xlarge, requiring cross-node deployment; for deployment, you can optionally use dynamoSpec with your own DGD or InferenceEndpointConfig, and the inference engine supports vLLM, SGLang, and TensorRT-LLM, with vLLM as the default.
AI Trends