←Search results

Search “AWS”

By relevance · Total 224 results Clear search

Saturday

  1. Wallstreetcn

    Amazon CEO Jassy: AWS's AI business has an annualized revenue of over $25 billion, and $220 billion in capital expenditure for 2026 is still not enough

    Amazon CEO Andy Jassy said on the a16z podcast that in the first three years of the generative AI wave, AWS's AI business has reached an annualized revenue run rate of over $25 billion, hundreds of times AWS's roughly $60 million in its early days. He said that under capital expenditure of about $220 billion for 2026, capacity for 2026 is already sold out, 2027 is essentially sold out, and a considerable portion of 2028 has also been sold, and that inference costs are still an order of magnitude too expensive, which is the main obstacle to widespread enterprise adoption of AI. He also mentioned that the Bedrock team rebuilt the inference engine with 6 engineers in 76 days, a task that previously required 40 people for a year.

    Amazon CEO Andy Jassy said on the a16z podcast that in the first three years of the generative AI wave, AWS's AI business has reached an annualized revenue run rate of over $25 billion, hundreds of times AWS's roughly $60 million in its early days. He said that under capital expenditure of about $220 billion for 2026, capacity for 2026 is already sold out, 2027 is essentially sold out, and a considerable portion of 2028 has also been sold, and that inference costs are still an order of magnitude

    Matched fieldsTitle

    #chinese#Industry news#English text is being prepared.#English text is being prepared.
  2. AWS Machine Learning

    Orchestrating SageMaker AI endpoint auto scaling with AWS CDK

    AWS released a reusable CDK construct that uses CloudFormation custom resources to automatically deregister and re-register Application Auto Scaling scaling targets before and after SageMaker AI endpoint updates, enabling instance type, instance count, or environment variable changes to be completed within a single deployment. The deployment stack contains 3 custom resources, 2 Step Functions state machines, and 12 Lambda functions, of which 7 are generated by the CDK provider framework.

    Four-step orchestration the construct runs during an endpoint update: deregister the scalable target, update the endpoint, register the scalable target, and attach the scaling policy

    Matched fieldsTitle

    #lab#Tutorials / practice#English text is being prepared.#Open-source ecosystem

Friday

  1. Hugging Face

    Hugging Face launches a Space to check whether models can run on AWS Trainium

    Hugging Face releases the torch-neuronx model support Space, which lets users view, by download count or trend, whether models on the Hub can run on AWS Trainium, and reports the number of sharded cores, compile-inclusive latency, and peak memory. Currently it only tests Transformers text-generation and image-text models, and only models that fit on 4 Trainium2 devices (up to 16 cores); each model generates 32 tokens for validation, comparing against CPU results in single-core mode, and in sharded mode a small model judges whether the English is fluent. Results come from the public dataset aws-neuron/torch-neuronx-model-support, and the Space also provides run scripts and a NEFF cache command that skips compilation.

    Hugging Face releases the torch-neuronx model support Space, which lets users view, by download count or trend, whether models on the Hub can run on AWS Trainium, and reports the number of sharded cores, compile-inclusive latency, and peak memory. Currently it only tests Transformers text-generation and image-text models, and only models that fit on 4 Trainium2 devices (up to 16 cores); each model generates 32 tokens for validation, comparing against CPU results in single-core mode, and in shard

    Matched fieldsTitle

    #lab#Tutorials / practice#English text is being prepared.#Open-source ecosystem
  2. Amazon Science

    Amazon releases TorchNeuron: a PyTorch backend for AWS Trainium

    This week, Amazon announced the open-sourcing of TorchNeuron, a backend that converts PyTorch code into operations executable on AWS Trainium; PyTorch code already running on other accelerators can be migrated with almost no changes. It offers an optional adaptive eager mode that groups operations for compilation and caches NEFF at multiple levels, cutting the time to the first training step by about one-third in cross-accelerator large language model tests. The team also used an AI agent to scan Hugging Face repositories, selecting 841 representative models and tens of thousands of tensor shapes to build a tiered testing process.

    This week, Amazon announced the open-sourcing of TorchNeuron, a backend that converts PyTorch code into operations executable on AWS Trainium; PyTorch code already running on other accelerators can be migrated with almost no changes. It offers an optional adaptive eager mode that groups operations for compilation and caches NEFF at multiple levels, cutting the time to the first training step by about one-third in cross-accelerator large language model tests. The team also used an AI agent to sca

    Matched fieldsTitle

    #company#Product updates#English text is being prepared.#English text is being prepared.
  3. AWS Machine LearningSelected

    AWS launches training sessions GA for SageMaker AI serverless model customization

    AWS announced that training sessions for SageMaker AI serverless model customization are generally available; users can write and orchestrate their own training loops, including custom loss functions and reward signals, while SageMaker AI handles GPU allocation, distributed execution, fault recovery, and checkpoint storage. The feature supports SFT, DPO, RLVR, RLAIF, and multi-turn RL, with a choice of LoRA or full fine-tuning, and currently supports gpt-oss-20b, gemma-4-31B-it, Qwen3.6-27B, and Qwen3.8-27B. Billing is per token processed by SageMaker AI, billing stops when the reward function scores in the user's environment, and no reserved capacity is required.

    AWS announced that training sessions for SageMaker AI serverless model customization are generally available; users can write and orchestrate their own training loops, including custom loss functions and reward signals, while SageMaker AI handles GPU allocation, distributed execution, fault recovery, and checkpoint storage. The feature supports SFT, DPO, RLVR, RLAIF, and multi-turn RL, with a choice of LoRA or full fine-tuning, and currently supports gpt-oss-20b, gemma-4-31B-it, Qwen3.6-27B, and

    Matched fieldsTitle

    #lab#Product updates#English text is being prepared.#English text is being prepared.

Thursday

  1. LMSYS Blog

    LMSYS Releases SGLang AWS Trainium Plugin Beta

    LMSYS, in collaboration with AWS, has released the SGLang Trainium plugin Beta, built on the newly released TorchNeuron, supporting LLM inference serving and RL rollout generation on Trainium2 and Trainium3. Initial coverage includes Qwen3 (dense), Qwen3-MoE-30B-A3B, DeepSeek-V2-Lite, GLM-4, and GLM-5.2/5.3, and supports data parallelism, tensor parallelism, expert parallelism, Radix Cache, BF16/FP8, and asynchronous scheduling. In the first end-to-end heterogeneous RL run, SGLang generated rollouts for Qwen3-MoE-30B-A3B on Trainium3, while Miles trained a GRPO policy with Megatron on a single H200 node. The Beta is currently limited to a small number of teams.

    LMSYS, in collaboration with AWS, has released the SGLang Trainium plugin Beta, built on the newly released TorchNeuron, supporting LLM inference serving and RL rollout generation on Trainium2 and Trainium3. Initial coverage includes Qwen3 (dense), Qwen3-MoE-30B-A3B, DeepSeek-V2-Lite, GLM-4, and GLM-5.2/5.3, and supports data parallelism, tensor parallelism, expert parallelism, Radix Cache, BF16/FP8, and asynchronous scheduling. In the first end-to-end heterogeneous RL run, SGLang generated roll

    Matched fieldsTitle

    #company#Product updates#English text is being prepared.#Open-source ecosystem
  2. AWS Machine Learning

    AWS team open-sources multi-agent system Deep Insight, breaking down five major challenges in production environments

    The AWS Solutions Architecture team built and open-sourced the multi-agent system Deep Insight, which can transform CSV data and natural language prompts into cited .docx analysis reports. The system consists of eight agents—Coordinator, Planner, Supervisor, Coder, Validator, Reporter, Tracker, and others—collaborating in layers, controlling execution flow based on the Graph and Agents-as-Tools patterns of the Strands Agents SDK, and selecting Claude Haiku, Opus, and Sonnet by role. Production deployment relies on the Amazon Bedrock AgentCore runtime; LLM-generated code is executed in isolation within per-session ephemeral containers on AWS Fargate, accompanied by a Web UI and an Ops dashboard.

    Deep Insight architecture: the Strands Agents multi-agent layer on Amazon Bedrock AgentCore, with an Application Load Balancer routing code execution to AWS Fargate containers in a private VPC

    The AWS Solutions Architecture team built and open-sourced the multi-agent system Deep Insight, which can transform CSV data and natural language prompts into cited .docx analysis reports. The system consists of eight agents—Coordinator, Planner, Supervisor, Coder, Validator, Reporter, Tracker, and others—collaborating in layers, controlling execution flow based on the Graph and Agents-as-Tools patterns of the Strands Agents SDK, and selecting Claude Haiku, Opus, and Sonnet by role. Production d

    Matched fieldsTitle

    #lab#Tutorials / practice#Agents#English text is being prepared.
  3. AWS Machine Learning

    How AWS PrivateLink Lets Amazon Quick Securely Access Data from Enterprise Intranets

    AWS has released a solution that uses AWS PrivateLink VPC endpoints and VPC connections to let Amazon Quick access on-premises data sources behind enterprise firewalls without traversing the public internet. User and API traffic goes through VPC endpoints, while data retrieval traffic goes through VPC connections, with security groups and endpoint policies layered on for defense in depth. VPC connections require an Amazon Quick Enterprise subscription, and Quick website VPC endpoints are not supported in AWS China Regions or the AWS GovCloud (US) Region.

    Architecture diagram of private access to Amazon Quick from a corporate data center through a VPC endpoint over AWS Direct Connect

    AWS has released a solution that uses AWS PrivateLink VPC endpoints and VPC connections to let Amazon Quick access on-premises data sources behind enterprise firewalls without traversing the public internet. User and API traffic goes through VPC endpoints, while data retrieval traffic goes through VPC connections, with security groups and endpoint policies layered on for defense in depth. VPC connections require an Amazon Quick Enterprise subscription, and Quick website VPC endpoints are not sup

    Matched fieldsTitle

    #lab#Tutorials / practice#English text is being prepared.#RAG
  4. AWS Machine Learning

    Training Qwen3 on AWS Trainium with torchtitan

    AWS has published a tutorial demonstrating three ways to train Qwen3-4B-Instruct-2507 with torchtitan on a single trn2.48xlarge (16 Trainium2 chips, 128 physical NeuronCores): Transformers v5 as the modeling backend, torchtitan's native Qwen3, and integration with the NKI MoE kernel. All three configurations share the same torchrun launch method and run on 64 logical NeuronCores; when loading from the released weights, the first-step loss is 6.54, while with random initialization it is 12.43.

    Training output showing a step 1 loss of 6.54 and the exported checkpoint loading with no missing or unexpected keys

    Matched fieldsTitle

    #lab#Tutorials / practice#English text is being prepared.#Open-source ecosystem
  5. Hugging Face

    Profiling TorchNeuron (Part 1): System-Level Performance Profiling on AWS Trainium

    AWS has released Part 1 of the TorchNeuron profiling guide, explaining how to use NeuronConfig and NeuronProfiler to collect system-level traces on AWS Trainium and view device timelines through Neuron Explorer. Taking RMSNorm as an example, a naive PyTorch implementation takes a total of 2.355 ms on the Neuron device, while the fused torch.rms_norm requires only 403.3 μs.

    AWS has released Part 1 of the TorchNeuron profiling guide, explaining how to use NeuronConfig and NeuronProfiler to collect system-level traces on AWS Trainium and view device timelines through Neuron Explorer. Taking RMSNorm as an example, a naive PyTorch implementation takes a total of 2.355 ms on the Neuron device, while the fused torch.rms norm requires only 403.3 μs.

    Matched fieldsTitle

    #lab#Tutorials / practice#English text is being prepared.#English text is being prepared.
  6. Hugging Face

    Profiling TorchNeuron (Part 1): System-Level Performance Profiling on AWS Trainium

    AWS has released Part 1 of the TorchNeuron profiling guide, which explains how to use NeuronConfig and NeuronProfiler for system-level profiling on AWS Trainium, and how to view system traces, device timelines, and per-engine breakdowns through Neuron Explorer. Taking RMSNorm as an example, a naive PyTorch implementation has a total device time of 2.355 ms, while torch.rms_norm(), fused into a single Neuron kernel, reduces this to 403.3 μs.

    AWS has released Part 1 of the TorchNeuron profiling guide, which explains how to use NeuronConfig and NeuronProfiler for system-level profiling on AWS Trainium, and how to view system traces, device timelines, and per-engine breakdowns through Neuron Explorer. Taking RMSNorm as an example, a naive PyTorch implementation has a total device time of 2.355 ms, while torch.rms norm(), fused into a single Neuron kernel, reduces this to 403.3 μs.

    Matched fieldsTitle

    #lab#Tutorials / practice#English text is being prepared.#English text is being prepared.

Wednesday

  2. AWS Machine Learning

    i14 Trains and Deploys the Laplace 3B Linear Attention Model on AWS Trainium2 Using Native PyTorch

    i14 partnered with AWS to train and deploy Laplace 3B, based on Gated Linear Attention, on AWS Trainium2 using TorchNeuron Native. A single trn2.48xlarge achieves a training throughput of about 120,000 tokens/s. The model's recurrent state per sequence is fixed at 29.5 MB and does not grow with context length; inference reaches 11,544 output tokens/s at 1,280 concurrent sequences. The team also reworked Muon's all_to_all communication to cut per-GPU communication volume by 24x and raise training throughput by 1.63x.

    NeuronCore-v3 internal architecture

    i14 partnered with AWS to train and deploy Laplace 3B, based on Gated Linear Attention, on AWS Trainium2 using TorchNeuron Native. A single trn2.48xlarge achieves a training throughput of about 120,000 tokens/s. The model's recurrent state per sequence is fixed at 29.5 MB and does not grow with context length; inference reaches 11,544 output tokens/s at 1,280 concurrent sequences. The team also reworked Muon's all to all communication to cut per-GPU communication volume by 24x and raise training

    Matched fieldsTitle

    #lab#Tutorials / practice#English text is being prepared.#English text is being prepared.
  4. AWS Machine Learning

    How AWS Uses X.509 PKI to Secure AI Agent Identity

    AWS and Keyfactor jointly released a reference architecture that uses Keyfactor EJBCA's X.509 PKI to establish verifiable, revocable cryptographic identities for AI agents, filling the gap left by MCP's lack of a built-in authentication standard. The solution combines Amazon Bedrock, the AWS Strands Agents SDK, AWS Secrets Manager, and Amazon RDS, and covers three trust boundaries—user authentication, agent-to-agent communication, and MCP tool invocation—through four layers of mechanisms: signed request envelopes, user-signed delegation assertions, RFC 8693 audience-restricted tokens, and real-time OCSP revocation checks.

    Three-layer PKI authentication architecture showing credential flow between the user, agents, and the MCP tool server

    AWS and Keyfactor jointly released a reference architecture that uses Keyfactor EJBCA's X.509 PKI to establish verifiable, revocable cryptographic identities for AI agents, filling the gap left by MCP's lack of a built-in authentication standard. The solution combines Amazon Bedrock, the AWS Strands Agents SDK, AWS Secrets Manager, and Amazon RDS, and covers three trust boundaries—user authentication, agent-to-agent communication, and MCP tool invocation—through four layers of mechanisms: signed

    Matched fieldsTitle

    #lab#Tutorials / practice#Safety / alignment#Agents
  5. AWS Machine Learning

    AWS Publishes Agent Registry Runtime Tool Discovery Practice

    AWS introduces the Agent Registry runtime tool discovery approach: agents use hybrid semantic and keyword search to find approved MCP servers and A2A agents, then instantly build clients from descriptors to execute requests. The sample code is based on Amazon Bedrock AgentCore Gateway and runtime, integrates an order management MCP server and two A2A agents for pricing and customer service, and demonstrates a return and refund scenario using pure semantic matching. The registry returns only connection metadata; invocation credentials must be obtained separately through Cognito OAuth or IAM, and new capabilities require no changes to orchestrator code.

    Comparison of hardcoded tool endpoints versus runtime discovery with AWS Agent Registry

    AWS introduces the Agent Registry runtime tool discovery approach: agents use hybrid semantic and keyword search to find approved MCP servers and A2A agents, then instantly build clients from descriptors to execute requests. The sample code is based on Amazon Bedrock AgentCore Gateway and runtime, integrates an order management MCP server and two A2A agents for pricing and customer service, and demonstrates a return and refund scenario using pure semantic matching. The registry returns only conn

    Matched fieldsTitle

    #lab#Tutorials / practice#Agents#English text is being prepared.

Tuesday

  1. Tessl: Product and Engineering Blog

    From Personal Coding Tool to Development Platform: AWS Kiro Product Lead on Trust and Cost in AI Coding

    Massimo Re Ferre, Senior Principal Product Manager at AWS, shared at DevCon: the speed at which teams delegate authority to AI agents depends on the cost of verifying their output, not on improvements in model capability. Agents have evolved from line-level completion to autonomously selecting tasks, and costs have shifted from fixed per-seat pricing to per-task consumption billing; about 80% of developers spend only 20% of the budget, while a small number of heavy users consume the rest.

    Massimo Re Ferre'

    Matched fieldsTitle

    #company#English text is being prepared.#Agents#English text is being prepared.