OpenRouter announced that it will conduct child sexual abuse material (CSAM) screening on all inference requests during request processing; content flagged as suspected CSAM may be retained for manual review and reported and preserved in accordance with the law. Screening includes comparing content against hashes of known CSAM, and suspected content that does not match a hash but is flagged by classifiers will also be retained. This screening also applies to requests covered by OpenRouter's zero data retention (ZDR) policy; ZDR does not exempt the obligations to report and preserve in accordance with the law.
OpenRouter announced that it will conduct child sexual abuse material (CSAM) screening on all inference requests during request processing; content flagged as suspected CSAM may be retained for manual review and reported and preserved in accordance with the law. Screening includes comparing content against hashes of known CSAM, and suspected content that does not match a hash but is flagged by classifiers will also be retained. This screening also applies to requests covered by OpenRouter's zero
Google A2UI adds swappable inference formats, among which A2UI Express, as the new default format, can reduce latency by up to 3x and cut the tokens needed to generate comparable UIs by up to 3x; when combined with decision models such as Jev for layout decisions, latency reductions can reach up to 80x. Express is now available in the Python agent SDK (a2ui-agent-sdk 0.9.0), the TypeScript SDK (@a2ui/agent), and the Dart SDK (a2ui_agent 0.0.1-wip004); developers can switch formats with a single parameter when creating a generator.
Google A2UI adds swappable inference formats, among which A2UI Express, as the new default format, can reduce latency by up to 3x and cut the tokens needed to generate comparable UIs by up to 3x; when combined with decision models such as Jev for layout decisions, latency reductions can reach up to 80x. Express is now available in the Python agent SDK (a2ui-agent-sdk 0.9.0), the TypeScript SDK (@a2ui/agent), and the Dart SDK (a2ui agent 0.0.1-wip004); developers can switch formats with a single
OpenAI has released GPT-6.1 Terra, which the company says combines detailed visual reasoning with precise computer operation and offers clearer, more natural writing, a marked improvement over GPT-5.6 Terra. The model is priced at $1 per million input tokens and $5 per million output tokens, roughly half the price of GPT-5.6 Terra and GPT-6.1 Sol, and about 10 times that of GPT-6 Luna. Simon Willison notes that Terra's position in OpenAI's model lineup is similar to Anthropic's Sonnet—neither the cheapest nor the most expensive—making it harder to judge when choosing.
OpenAI has released GPT-6.1 Terra, which the company says combines detailed visual reasoning with precise computer operation and offers clearer, more natural writing, a marked improvement over GPT-5.6 Terra. The model is priced at $1 per million input tokens and $5 per million output tokens, roughly half the price of GPT-5.6 Terra and GPT-6.1 Sol, and about 10 times that of GPT-6 Luna. Simon Willison notes that Terra's position in OpenAI's model lineup is similar to Anthropic's Sonnet—neither th
SGLang matches or exceeds DeepInfer on DeepInfer's own benchmarks: up to 1.04x prefill throughput on 4x GB300 and 4x B300, and 1.1%–7.4% shorter decode step time on 64x GB300. SGLang has integrated DeepSeek's Mega series kernels, DSpark speculative decoding, SWA bounded replay, and two Engram placement methods, and has filled in KV cache, prefill/decode disaggregation, and inter-replica load balancing that DeepInfer does not include. SGLang supports NVIDIA GB300, B300, B200, H200, and AMD MI355X.
SGLang matches or exceeds DeepInfer on DeepInfer's own benchmarks: up to 1.04x prefill throughput on 4x GB300 and 4x B300, and 1.1%–7.4% shorter decode step time on 64x GB300. SGLang has integrated DeepSeek's Mega series kernels, DSpark speculative decoding, SWA bounded replay, and two Engram placement methods, and has filled in KV cache, prefill/decode disaggregation, and inter-replica load balancing that DeepInfer does not include. SGLang supports NVIDIA GB300, B300, B200, H200, and AMD MI355X
Amazon CEO Andy Jassy said in an interview on the a16z podcast that the company's capital expenditure is expected to reach $220 billion in 2026, but compute capacity is already sold out for 2026, essentially fully booked for 2027, and a large share of 2028 quotas has been locked in. He described enterprise AI demand as dumbbell-shaped: a large number of production systems in the middle have yet to be deeply integrated because inference costs are too high, and costs need to fall by another order of magnitude before an explosion in adoption occurs. Jassy also noted that 99% of AWS's top 1,000 customers use Graviton, which offers about 40% better price-performance than x86, while Trainium has a 30% to 40% advantage over comparable hardware.
Amazon CEO Andy Jassy said in an interview on the a16z podcast that the company's capital expenditure is expected to reach $220 billion in 2026, but compute capacity is already sold out for 2026, essentially fully booked for 2027, and a large share of 2028 quotas has been locked in. He described enterprise AI demand as dumbbell-shaped: a large number of production systems in the middle have yet to be deeply integrated because inference costs are too high, and costs need to fall by another order
Sebastian Raschka has released 8 programming videos on building a reasoning model from scratch, totaling 11 hours, using Python and PyTorch to implement evaluation, inference-time scaling, reinforcement learning, and distillation starting from a pretrained Qwen3 0.6B model. The course first builds a math answer verifier and evaluates on MATH-500, then implements temperature sampling, top-p filtering, and self-consistency, followed by RLVR training with a simplified version of GRPO, and finally distills a small model using answers generated by DeepSeek R1. The content accompanies his book "Build a Reasoning Model (From Scratch)"; the early text generation and evaluation examples can run on a CPU, while later training experiments are recommended to use a GPU.
Sebastian Raschka has released 8 programming videos on building a reasoning model from scratch, totaling 11 hours, using Python and PyTorch to implement evaluation, inference-time scaling, reinforcement learning, and distillation starting from a pretrained Qwen3 0.6B model. The course first builds a math answer verifier and evaluates on MATH-500, then implements temperature sampling, top-p filtering, and self-consistency, followed by RLVR training with a simplified version of GRPO, and finally d
In an a16z interview, Amazon CEO Andy Jassy said AI inference costs are still an order of magnitude too high, and that for AI to be widely used in enterprise software, costs need to fall to about one-tenth of current levels. He said Amazon plans to spend $220 billion in capital expenditure this year, that its main compute supply for 2026 is essentially fully booked, and that most of 2027 has also been reserved; the in-house Trainium chip offers about 30% to 40% better price-performance than comparable processors, and has already secured more than $225 billion in future revenue commitments.
In an a16z interview, Amazon CEO Andy Jassy said AI inference costs are still an order of magnitude too high, and that for AI to be widely used in enterprise software, costs need to fall to about one-tenth of current levels. He said Amazon plans to spend $220 billion in capital expenditure this year, that its main compute supply for 2026 is essentially fully booked, and that most of 2027 has also been reserved; the in-house Trainium chip offers about 30% to 40% better price-performance than comp
Codex developers get an experience update: reasoning levels can be switched mid-conversation without invalidating the cache; use low reasoning for simple tasks and raise it when needed, with no need to rebuild context. The CLI is updated in sync, including reusable prompt presets, inline switching of models and permissions, long-session resume fixes, enterprise MCP login and Windows proxy authentication, and noticeably faster dot invocation connection speeds.
Artificial Analysis published preliminary test results on AA-AgentPerf for Meta's new open-weight model Muse Spark Open, an 816B-parameter, 42B-active MoE that scores 39 on the Artificial Analysis Intelligence Index. Tests show NVIDIA B300 and GB300 each reach about 150M tokens/s per megawatt under throughput-first configurations, with GB300 at about 108M and B300 at about 92M at 100 tok/s output speed; AMD MI355X reaches 70M tokens/s per megawatt at 65 tok/s. 46 of the model's 62 layers use 2,048-token sliding window attention, reducing KV cache requirements at 1M context by about 75% versus full attention; Meta released an NVFP4 checkpoint and AMD released an MXFP4 checkpoint.
Artificial Analysis published preliminary test results on AA-AgentPerf for Meta's new open-weight model Muse Spark Open, an 816B-parameter, 42B-active MoE that scores 39 on the Artificial Analysis Intelligence Index. Tests show NVIDIA B300 and GB300 each reach about 150M tokens/s per megawatt under throughput-first configurations, with GB300 at about 108M and B300 at about 92M at 100 tok/s output speed; AMD MI355X reaches 70M tokens/s per megawatt at 65 tok/s. 46 of the model's 62 layers use 2,0
Nathan Lambert says DeepSeek processes 150T tokens per day on a single cluster, more than the combined open-source model inference volume on OpenRouter, Fireworks, Together, and BaseTen. He also posted a reconstructed comparison chart of his own based on these companies' inference volume reports, and noted that this figure corresponds to only a single cluster; it remains unclear how many clusters DeepSeek actually has.
MiniMax announced a partnership with Inferact to jointly advance production inference for MiniMax models, with the related engineering improvements contributed back to the vLLM project. MiniMax also thanked the Inferact team for supporting M3.1 Flash in production from day one.
Cohere has launched confidential computing capabilities on Model Vault, currently in public beta, running on NVIDIA GPUs on Microsoft Azure. The feature combines confidential CPU and GPU execution, OHTTP end-to-end encryption of request content, and remote attestation, allowing customers to verify the target environment before releasing sensitive data. Cohere plans to open-source the confidential inference stack and advance support for NVIDIA Blackwell and B200 GPUs.
MiniMax officially responded to SGLang adding MiniMax H3 to the TPU support list of SGLang-TorchTPU, and said it looks forward to more open-source work on efficient video inference. SGLang said that SGLang-TorchTPU was built jointly by SGLang, the Google TPU team, and the Meta PyTorch team, has been used for production inference at Meta, and supports models including Muse Spark Open, GLM-5.3 Flash, Kimi K3, Ling 3.1 Flash, and MiniMax H3.
VocabX selects tokens by cost rather than frequency and simultaneously rebuilds subword segmentation and multi-word merging, compressing the vocabulary of domain-specific small models to a smaller scale and reducing inference costs by 77% in real-world applications. Taking a scoring model as an example, generating one token costs 7.4 times as much as reading one token, and the output-layer vocabulary shrinks from about 152,000 rows to about 65,000 rows. This method targets the memory bandwidth bottleneck of small models such as Qwen3-4B at medium batch sizes, whereas methods such as FVT, MWT, and in-place BPE select tokens only by frequency and address only a single level.
VocabX selects tokens by cost rather than frequency and simultaneously rebuilds subword segmentation and multi-word merging, compressing the vocabulary of domain-specific small models to a smaller scale and reducing inference costs by 77% in real-world applications. Taking a scoring model as an example, generating one token costs 7.4 times as much as reading one token, and the output-layer vocabulary shrinks from about 152,000 rows to about 65,000 rows. This method targets the memory bandwidth b
DeepSeek has open-sourced its internal inference framework Deepinfer on GitHub, with the goal of reducing the compute and energy costs of deploying models on your own. During the inference stage, the framework distributes load more evenly across available processor cores, uses speculative decoding to speed up responses, and improves the structure of results before output; it currently supports only DeepSeek-V4.1-Flash, with more of its own models to be opened up later. On the hardware side, it requires Nvidia GPUs with the SM100 architecture or Huawei Ascend 950 NPUs, so it is not aimed at individual users.
DeepSeek has open-sourced its internal inference framework Deepinfer on GitHub, with the goal of reducing the compute and energy costs of deploying models on your own. During the inference stage, the framework distributes load more evenly across available processor cores, uses speculative decoding to speed up responses, and improves the structure of results before output; it currently supports only DeepSeek-V4.1-Flash, with more of its own models to be opened up later. On the hardware side, it r
DeepSeek has open-sourced its internal inference framework DeepInfer, which it says powers the high-throughput, low-latency inference of DeepSeek-V4.1-Flash. The framework supports both Nvidia and Huawei Ascend hardware, and offers capabilities such as distributed inference, speculative decoding, SWA Bounded Replay, and structured output. DeepSeek also released deepseek-recipe, a production-grade tokenization component fully aligned with the official API.
