Search “RAG”

No More Guesswork: A Measured Approach to TTS Autoscaling Targets on Amazon SageMaker AI

AWS deploys the 82M-parameter TTS model Kokoro-82M in a custom container on an Amazon SageMaker AI real-time endpoint, using measured Q_max (the highest concurrency a single instance can sustain within the SLO) and T_total (the end-to-end time of one scale-out) to derive the scale-out threshold, scale-in threshold, and cooldown period, replacing target values set by guesswork. The approach scales on the natively published SageMaker AI ConcurrentRequestsPerModel metric, avoiding the hidden failure of a custom metric that no one reports.

LMSYS Releases SGLang AWS Trainium Plugin Beta

LMSYS, in collaboration with AWS, has released the SGLang Trainium plugin Beta, built on the newly released TorchNeuron, supporting LLM inference serving and RL rollout generation on Trainium2 and Trainium3. Initial coverage includes Qwen3 (dense), Qwen3-MoE-30B-A3B, DeepSeek-V2-Lite, GLM-4, and GLM-5.2/5.3, and supports data parallelism, tensor parallelism, expert parallelism, Radix Cache, BF16/FP8, and asynchronous scheduling. In the first end-to-end heterogeneous RL run, SGLang generated rollouts for Qwen3-MoE-30B-A3B on Trainium3, while Miles trained a GRPO policy with Megatron on a single H200 node. The Beta is currently limited to a small number of teams.

Developers build all-new app experiences for the first foldable iPhone Duo

Apple announced that developers are optimizing apps and games for the first foldable iPhone Duo, which goes on sale on October 23 (this Friday). The App Store will feature a dedicated editorial collection and will identify compatible apps with a Made for iPhone Duo badge. The iPhone Duo's inner display is 50% larger than that of the iPhone 18 Pro Max, is powered by the A20 Pro, and supports Split View for the first time on an iPhone. Netflix, Prime Video, Paramount+, Zoom, Slack, and others are already compatible, while Microsoft Word and Outlook will deliver desktop-class experiences in the coming weeks.

MiniMax Code Desktop Launches Mini App

MiniMax has launched Mini App in local sessions of the MiniMax Code desktop app (macOS / Windows). It is a persistent application with a frontend and a Node.js backend, installed as a plugin, that can call and be called by the Agent, keeping work on a continuously alive page rather than scattered across conversations. This release includes the Mini App runtime (persistent data directory, hosted MCP, Agent API, Host Connectors, and in-page tool approval), /miniapp-creator for generating a page with a single sentence, and the official Mini Apps in the plugin marketplace and the open-source community repository MiniMax-Code-MiniApps. According to the official announcement, account credentials are kept by the host and cannot be accessed by plugins or pages, and content passed from a page to the Agent first enters the input box for review.

Thursday Tech Briefing: Data Protection Issues in AfD Email Leak, Meta Platforms Profits Soar

Meta Platforms' third-quarter revenue and net profit grew substantially, with net profit increasing more than twofold, daily active users reaching 3.64 billion, and headcount down 15% year-over-year. A New Mexico court found Facebook engaged in systematic deceptive practices and sentenced Meta Platforms to a $2.5 billion fine. AfD was hacked, with over 400,000 emails and 50,000 documents leaked; whether the party has reported to data protection authorities remains unclear.

India's Ministry of Information and Broadcasting asks TV channels to limit AI to "auxiliary uses," does not clarify whether AI anchors are prohibited

India's Ministry of Information and Broadcasting (MIB) issued an advisory on October 21 requiring private satellite television channels to restrict the use of AI and generative AI to "auxiliary uses," and listed "virtual news anchoring" as a scenario requiring caution, but did not define what constitutes auxiliary use, nor did it clarify whether it covers the AI anchors used by channels such as Aaj Tak, News18 and Odisha TV. The advisory did not require channels to label AI anchors, nor did it cite specific provisions of the Programme Code; Doordarshan, which is under MIB, itself also uses AI anchors, and in March this year told a parliamentary committee that it hoped to expand their use.

Google CEO Pichai: The advantage of AI agents is that they work 24 hours a day, 7 days a week

Google CEO Sundar Pichai has handed off information gathering to AI agents, which compile developments from the past 24 hours, summarize progress on Google's AI model research and compute resource usage, and follow up on discussions across multiple internal chat groups. He said AI agents work continuously in the cloud 24 hours a day, 7 days a week, can understand prior work progress, and pick up where they left off. The report also noted that an engineering director at Cisco who manages more than 10 AI agents said supervising these agents requires constant task switching, and working hours have not been shortened.

Alloop three-in-one AI health wearable: wristband, ring and Cube sensor in interchangeable combinations

Alloop has launched Trio, a modular AI-native health wearable consisting of a wristband, a ring, a Cube sensor and a Halo, which can be combined into either a wristband or ring form. It measures resting heart rate, heart rate variability, blood oxygen, respiratory rate and body temperature, and provides a sleep score and Bio Age. Its AI health agent calls Google Gemini and ChatGPT; the app is subscription-free, with an optional Pro plan at about $5/month. Trio will launch on Kickstarter on November 10 at $299, with shipping expected between late December 2026 and early January 2027.

AWS team open-sources multi-agent system Deep Insight, breaking down five major challenges in production environments

The AWS Solutions Architecture team built and open-sourced the multi-agent system Deep Insight, which can transform CSV data and natural language prompts into cited .docx analysis reports. The system consists of eight agents—Coordinator, Planner, Supervisor, Coder, Validator, Reporter, Tracker, and others—collaborating in layers, controlling execution flow based on the Graph and Agents-as-Tools patterns of the Strands Agents SDK, and selecting Claude Haiku, Opus, and Sonnet by role. Production deployment relies on the Amazon Bedrock AgentCore runtime; LLM-generated code is executed in isolation within per-session ephemeral containers on AWS Fargate, accompanied by a Web UI and an Ops dashboard.

How AWS PrivateLink Lets Amazon Quick Securely Access Data from Enterprise Intranets

AWS has released a solution that uses AWS PrivateLink VPC endpoints and VPC connections to let Amazon Quick access on-premises data sources behind enterprise firewalls without traversing the public internet. User and API traffic goes through VPC endpoints, while data retrieval traffic goes through VPC connections, with security groups and endpoint policies layered on for defense in depth. VPC connections require an Amazon Quick Enterprise subscription, and Quick website VPC endpoints are not supported in AWS China Regions or the AWS GovCloud (US) Region.

Sierra launches 𝜏-i18n multilingual voice agent evaluation benchmark

Sierra releases 𝜏-i18n, extending 𝜏-Voice to five languages: Spanish, Brazilian Portuguese, Hindi, Korean, and Mandarin, retaining the same airline, retail, and telecom tasks, policies, tools, and success criteria to enable cross-language comparison. Across 2,700 main calls, task completion rates for Spanish, Hindi, and Portuguese lag English by an average of 2.4, 3.8, and 6.4 percentage points, while Korean and Mandarin lag by 22.6 and 9.1 percentage points respectively; Korean is the weakest language among the three agents. In addition to task completion, 𝜏-i18n separately reports interaction metrics (response timing, selectivity, tool-call efficiency) and generation quality (naturalness, speech fidelity), and has open-sourced the codebase and the language pack framework LanguageFactory.

Training Qwen3 on AWS Trainium with torchtitan

AWS has published a tutorial demonstrating three ways to train Qwen3-4B-Instruct-2507 with torchtitan on a single trn2.48xlarge (16 Trainium2 chips, 128 physical NeuronCores): Transformers v5 as the modeling backend, torchtitan's native Qwen3, and integration with the NKI MoE kernel. All three configurations share the same torchrun launch method and run on 64 logical NeuronCores; when loading from the released weights, the first-step loss is 6.54, while with random initialization it is 12.43.

TorchTitan Update: Support for Multiple New SOTA Models, TitanRL, and Large-Scale Performance Improvements

PyTorch's native training platform TorchTitan has released an update that adds full training-stack support for Muse Spark Open, Muse Glimmer, Qwen3-VL, Qwen3.5, Kimi K2.7, Kimi K3, and DeepSeek-V4, and introduces the reinforcement learning component TitanRL. TitanRL lets pretraining, post-training, and vLLM inference share the same model code, and supports asynchronous rollout as well as agent-task training such as long-context and code execution. On performance, FlexShard on 256 H100 GPUs reduces DeepSeekV3 optimizer time by up to 79%, and reduces end-to-end training step time on Llama3 405B by up to 20%.

PyTorch integrates FlyDSL backend on AMD GPUs

The official PyTorch blog introduces that users of AMD MI350 series GPUs can now use FlyDSL through existing operator APIs, covering dense and grouped GEMM, MXFP8/MXFP4 scaled GEMM, RMSNorm, TopK, and attention kernels provided via the AOTriton 0.14b series, without calling custom kernels. On MI355X, BF16 dense GEMM achieves a 1.10x geometric mean speedup over the faster baseline among ATen and Triton, MXFP8 and MXFP4 achieve 1.58x and 1.68x over ATen respectively, and grouped GEMM achieves 1.20x over Triton. In vLLM end-to-end A/B tests, whole-request speedups are BF16 1.13x, MXFP8 2.04x, and MXFP4 3.01x, depending on the model and concurrency.

NVIDIA Releases Kumo Relational Relational Foundation Model, Predicts Without Training

NVIDIA has released Kumo Relational, a pretrained relational foundation model that can make predictions directly on enterprise databases in a single forward pass, without task-specific training, feature engineering, or table flattening. Across 41 tasks on 15 databases—RelBench V1/V2, SAP SALT, and 4DBInfer—it outperforms supervised relational deep learning models and gradient-boosted tree pipelines built with expert feature engineering, achieving an 8% higher MRR than the AutoGluon ensemble on SAP SALT. The model is available through NVIDIA's open-source GPU-native structured-data-models library or NVIDIA NIM, and supports databases with 500B+ rows.

Will Larson on the revised rules of engineering management: How AI is changing engineering management and the engineer-to-CTO path

Imprint CTO Will Larson discussed on a podcast how AI and shifting industry expectations are reshaping engineering management, the career ladder, and the growth path from engineer to CTO, and advised middle managers to return to frontline management and relearn technical skills. Drawing on his experience at Digg, Yahoo, Stripe, Uber, Carta, and Calm, he discussed cases such as the Digg v4 rewrite and how Yahoo's server approval process spawned a "black market," as well as how AI is bringing technical decision-making power back to management.

Profiling TorchNeuron (Part 1): System-Level Performance Profiling on AWS Trainium

AWS has released Part 1 of the TorchNeuron profiling guide, explaining how to use NeuronConfig and NeuronProfiler to collect system-level traces on AWS Trainium and view device timelines through Neuron Explorer. Taking RMSNorm as an example, a naive PyTorch implementation takes a total of 2.355 ms on the Neuron device, while the fused torch.rms_norm requires only 403.3 μs.

English translation is being prepared.

Baseten Base Labs releases decision models mammoth-1 and elephant-1

Baseten's Base Labs releases two decision models, elephant-1 (9B) and mammoth-1 (27B), both fully fine-tuned from Qwen3.5, with model weights publicly released. mammoth-1 ranks first among 208 models on the Jev Decision Index 0.4 with a score of 71.24, leading the second-place Perplexity Decider v1.2 by 1.8 points; elephant-1 scores 62.65 on that leaderboard and ties for first with d1-ultra on the Artificial Analysis Decision Index with a score of 65. Training used the eighth version of the data mixture with 6.21M samples, and replaced the full-attention layers with a block bidirectional attention mask; under fixed data, this change raised the public 10k-sample score from 44.8 to 48.6.

English translation is being prepared.