Understand this story
On October 22, 2026, LMSYS partnered with AWS to release the SGLang Trainium plugin Beta. Built on the newly released TorchNeuron, the plugin supports LLM inference serving and RL rollout generation on Trainium2 and Trainium3. Initial coverage includes Qwen3 (dense), Qwen3-MoE-30B-A3B, DeepSeek-V2-Lite, GLM-4, and GLM-5.2/5.3, and it supports data parallelism, tensor parallelism, expert parallelism, Radix Cache, BF16/FP8, and asynchronous scheduling. In the first end-to-end heterogeneous RL run, SGLang generated rollouts for Qwen3-MoE-30B-A3B on Trainium3, while Miles trained a GRPO policy with Megatron on a single H200 node. The Beta is currently limited to a small number of teams. The next day, Amazon announced the open-sourcing of TorchNeuron, a backend that converts PyTorch code into operations executable on AWS Trainium, allowing PyTorch code already running on other accelerators to migrate with almost no changes.
AI-generated from public reports
Latest development · Report
Amazon releases TorchNeuron: a PyTorch backend for AWS TrainiumDevelopments
2 confirmed developments
- Source:Amazon Science
- Source:LMSYS Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 23
Amazon ScienceFirst-party
Amazon releases TorchNeuron: a PyTorch backend for AWS Trainium
This week, Amazon announced the open-sourcing of TorchNeuron, a backend that converts PyTorch code into operations executable on AWS Trainium; PyTorch code already running on other accelerators can be migrated with almost no changes. It offers an optional adaptive eager mode that groups operations for compilation and caches NEFF at multiple levels, cutting the time to the first training step by about one-third in cross-accelerator large language model tests. The team also used an AI agent to scan Hugging Face repositories, selecting 841 representative models and tens of thousands of tensor shapes to build a tiered testing process.
Oct 22
LMSYS BlogFirst-party
LMSYS Releases SGLang AWS Trainium Plugin Beta
LMSYS, in collaboration with AWS, has released the SGLang Trainium plugin Beta, built on the newly released TorchNeuron, supporting LLM inference serving and RL rollout generation on Trainium2 and Trainium3. Initial coverage includes Qwen3 (dense), Qwen3-MoE-30B-A3B, DeepSeek-V2-Lite, GLM-4, and GLM-5.2/5.3, and supports data parallelism, tensor parallelism, expert parallelism, Radix Cache, BF16/FP8, and asynchronous scheduling. In the first end-to-end heterogeneous RL run, SGLang generated rollouts for Qwen3-MoE-30B-A3B on Trainium3, while Miles trained a GRPO policy with Megatron on a single H200 node. The Beta is currently limited to a small number of teams.
Heat trend for this story
- Current
- 7
- Period peak
- 15 Oct 23 05:00
- 24-hour change
- -50%
Move the pointer or select the chart to inspect hourly heat. Use the left and right arrow keys with a keyboard.