Search “AWS”

AWS publishes tutorial on deriving autoscaling targets for TTS on SageMaker AI

On October 23, 2026, AWS published a tutorial introducing a method for deriving autoscaling targets for TTS on Amazon SageMaker AI. The tutorial deploys the 82M-parameter TTS model Kokoro-82M as a custom container on a SageMaker AI real-time endpoint, and uses measured Q_max (the maximum concurrency a single instance can sustain within the SLO) and T_total (the end-to-end time of one scale-out) to calculate the scale-out threshold, scale-in threshold, and cooldown period, replacing target values set by guesswork. The solution scales on the ConcurrentRequestsPerModel metric natively published by SageMaker AI, avoiding the hidden failure of custom metrics that no one reports. There are currently no further developments.

AWS publishes tutorial on deriving auto-scaling targets for TTS on SageMaker AI

On October 23, 2026, AWS published a tutorial introducing a method for deriving autoscaling targets for TTS on Amazon SageMaker AI. The tutorial deploys the 82M-parameter TTS model Kokoro-82M as a custom container on a SageMaker AI real-time endpoint, and uses measured Q_max (the maximum concurrency a single instance can sustain within the SLO) and T_total (the end-to-end time of one scale-out) to calculate the scale-out threshold, scale-in threshold, and cooldown period, replacing target values set by guesswork. The solution scales on the ConcurrentRequestsPerModel metric natively published by SageMaker AI, avoiding the hidden failure of custom metrics that no one reports. There are currently no further developments.

View story:AWS publishes tutorial on deriving autoscaling targets for TTS on SageMaker AI

LMSYS releases SGLang AWS Trainium plugin Beta

On October 22, 2026, LMSYS partnered with AWS to release the SGLang Trainium plugin Beta. Built on the newly released TorchNeuron, the plugin supports LLM inference serving and RL rollout generation on Trainium2 and Trainium3. Initial coverage includes Qwen3 (dense), Qwen3-MoE-30B-A3B, DeepSeek-V2-Lite, GLM-4, and GLM-5.2/5.3, and it supports data parallelism, tensor parallelism, expert parallelism, Radix Cache, BF16/FP8, and asynchronous scheduling. In the first end-to-end heterogeneous RL run, SGLang generated rollouts for Qwen3-MoE-30B-A3B on Trainium3, while Miles trained a GRPO policy with Megatron on a single H200 node. The Beta is currently limited to a small number of teams. The next day, Amazon announced the open-sourcing of TorchNeuron, a backend that converts PyTorch code into operations executable on AWS Trainium, allowing PyTorch code already running on other accelerators to migrate with almost no changes.

View story:LMSYS releases SGLang AWS Trainium plugin Beta

AWS Solutions Architecture team builds multi-agent system Deep Insight

The AWS Solutions Architecture team built and open-sourced the multi-agent system Deep Insight, which can transform CSV data and natural language prompts into cited .docx analysis reports. The system consists of eight agents—Coordinator, Planner, Supervisor, Coder, Validator, Reporter, Tracker, and others—collaborating in layers, controlling execution flow based on the Graph and Agents-as-Tools patterns of the Strands Agents SDK, and selecting Claude Haiku, Opus, and Sonnet respectively by role. Production deployment relies on the Amazon Bedrock AgentCore runtime; LLM-generated code is executed in isolation in per-session temporary containers on AWS Fargate, accompanied by a Web UI and Ops dashboard. Currently this fact is reported only by a single source, AWS Machine Learning, and there are no subsequent developments.

AWS Solutions Architecture team builds multi-agent system Deep Insight

The AWS Solutions Architecture team built and open-sourced the multi-agent system Deep Insight, which can transform CSV data and natural language prompts into cited .docx analysis reports. The system consists of eight agents—Coordinator, Planner, Supervisor, Coder, Validator, Reporter, Tracker, and others—collaborating in layers, controlling execution flow based on the Graph and Agents-as-Tools patterns of the Strands Agents SDK, and selecting Claude Haiku, Opus, and Sonnet respectively by role. Production deployment relies on the Amazon Bedrock AgentCore runtime; LLM-generated code is executed in isolation in per-session temporary containers on AWS Fargate, accompanied by a Web UI and Ops dashboard. Currently this fact is reported only by a single source, AWS Machine Learning, and there are no subsequent developments.

View story:AWS Solutions Architecture team builds multi-agent system Deep Insight

English translation is being prepared.

English translation is being prepared.

View story:English translation is being prepared.

i14 migrates its custom architecture to AWS Trainium2

i14 migrated its custom architecture to AWS Trainium2. According to information released by AWS Machine Learning on October 21, 2026, i14 collaborated with AWS to train and deploy the Gated Linear Attention-based Laplace 3B model on AWS Trainium2 using TorchNeuron Native. Training throughput on a single trn2.48xlarge is approximately 120,000 tokens/s; the model's recurrent state is fixed at 29.5 MB per sequence and does not grow with context length; inference reaches 11,544 output tokens/s at 1,280 concurrent sequences. The team also reworked Muon's all_to_all communication, reducing per-card communication volume by 24x and increasing training throughput by 1.63x. There are currently no subsequent developments.

View story:i14 migrates its custom architecture to AWS Trainium2

AWS presents: Securing AI agent identities with Keyfactor EJBCA and AWS services

On October 21, 2026, AWS presented a solution for securing AI agent identities using Keyfactor EJBCA combined with AWS services. According to an article published by AWS Machine Learning, AWS and Keyfactor jointly released a reference architecture that uses Keyfactor EJBCA's X.509 PKI to establish verifiable, revocable cryptographic identities for AI agents, filling the gap left by MCP's lack of a built-in authentication standard. The solution combines Amazon Bedrock, the AWS Strands Agents SDK, AWS Secrets Manager, and Amazon RDS, and covers three trust boundaries—user authentication, agent-to-agent communication, and MCP tool invocation—through four layers of mechanisms: signed request envelopes, user-signed delegation assertions, RFC 8693 audience-restricted tokens, and real-time OCSP revocation checks. Currently, the solution is presented publicly only as a reference architecture, and there are no public facts regarding subsequent implementation progress.

View story:AWS presents using Keyfactor EJBCA and AWS services to secure AI agent identities

English translation is being prepared.

i14 migrates its custom architecture to AWS Trainium2

i14 migrated its custom architecture to AWS Trainium2. According to information released by AWS Machine Learning on October 21, 2026, i14 collaborated with AWS to train and deploy the Gated Linear Attention-based Laplace 3B model on AWS Trainium2 using TorchNeuron Native. Training throughput on a single trn2.48xlarge is approximately 120,000 tokens/s; the model's recurrent state is fixed at 29.5 MB per sequence and does not grow with context length; inference reaches 11,544 output tokens/s at 1,280 concurrent sequences. The team also reworked Muon's all_to_all communication, reducing per-card communication volume by 24x and increasing training throughput by 1.63x. There are currently no subsequent developments.

AWS presents using Keyfactor EJBCA and AWS services to secure AI agent identities

On October 21, 2026, AWS presented a solution for securing AI agent identities using Keyfactor EJBCA combined with AWS services. According to an article published by AWS Machine Learning, AWS and Keyfactor jointly released a reference architecture that uses Keyfactor EJBCA's X.509 PKI to establish verifiable, revocable cryptographic identities for AI agents, filling the gap left by MCP's lack of a built-in authentication standard. The solution combines Amazon Bedrock, the AWS Strands Agents SDK, AWS Secrets Manager, and Amazon RDS, and covers three trust boundaries—user authentication, agent-to-agent communication, and MCP tool invocation—through four layers of mechanisms: signed request envelopes, user-signed delegation assertions, RFC 8693 audience-restricted tokens, and real-time OCSP revocation checks. Currently, the solution is presented publicly only as a reference architecture, and there are no public facts regarding subsequent implementation progress.

AWS introduces vLLM bidirectional KV transfer to reduce multi-turn TTFT

On October 20, 2026, AWS introduced a new bidirectional KV cache transfer mechanism in vLLM: in subsequent turns, the decode node sends the already-computed KV cache back to the prefill node, avoiding recomputation of the conversation history. On a 1P1D deployment on Amazon EC2 P5en, this mechanism improves TTFT for Qwen3-32B by up to 1.6x and for zai-org/GLM-5-FP8 by up to 3.3x. The capability was initially validated on AWS Trainium clusters and contributed to vLLM as an accelerator-agnostic feature that requires coordination with a KV-cache-aware router for cross-turn reuse. There are currently no further developments.

Deepgram and AWS port Nova-3 to Trainium2

Deepgram and AWS collaborated to port the Nova-3 speech-to-text model to AWS Trainium2. According to information published by AWS Machine Learning on October 20, 2026, a single trn2.3xlarge instance can run 8 model replicas, processing 11%–70% more audio per dollar in offline batch processing scenarios than the previous solution. The team used TorchNeuron to complete the PyTorch port and tuned it through NKILib fused kernels and hand-written NKI kernels, among which the FFN kernel latency was reduced by 2.0x, depthwise conv1d was sped up by 1.5x, and the conv2d kernel unlocked batch sizes above 32. There are currently no subsequent development facts.

AWS introduces vLLM bidirectional KV transfer to reduce multi-turn TTFT

On October 20, 2026, AWS introduced a new bidirectional KV cache transfer mechanism in vLLM: in subsequent turns, the decode node sends the already-computed KV cache back to the prefill node, avoiding recomputation of the conversation history. On a 1P1D deployment on Amazon EC2 P5en, this mechanism improves TTFT for Qwen3-32B by up to 1.6x and for zai-org/GLM-5-FP8 by up to 3.3x. The capability was initially validated on AWS Trainium clusters and contributed to vLLM as an accelerator-agnostic feature that requires coordination with a KV-cache-aware router for cross-turn reuse. There are currently no further developments.

View story:AWS introduces vLLM bidirectional KV transfer to reduce multi-turn TTFT

English translation is being prepared.

View story:English translation is being prepared.

Deepgram and AWS port Nova-3 to Trainium2

Deepgram and AWS collaborated to port the Nova-3 speech-to-text model to AWS Trainium2. According to information published by AWS Machine Learning on October 20, 2026, a single trn2.3xlarge instance can run 8 model replicas, processing 11%–70% more audio per dollar in offline batch processing scenarios than the previous solution. The team used TorchNeuron to complete the PyTorch port and tuned it through NKILib fused kernels and hand-written NKI kernels, among which the FFN kernel latency was reduced by 2.0x, depthwise conv1d was sped up by 1.5x, and the conv2d kernel unlocked batch sizes above 32. There are currently no subsequent development facts.

View story:Deepgram and AWS port Nova-3 to Trainium2

English translation is being prepared.

English translation is being prepared.

View story:English translation is being prepared.

English translation is being prepared.