Search “Agents”

Fangcun Lab releases an 87-page safety evaluation report on DeepSeek V4.1 Flash-1017

The third-party evaluation lab Fangcun (@FangcunLeap) from China has released an 87-page report evaluating DeepSeek V4.1 Flash-1017's performance on loss of control and harmful manipulation. The report finds that it shows clear safety progress compared with the DeepSeek V4 generation, approaching or even matching the top closed-source models tested in the report on points such as "not strategically hiding capabilities and still respecting approval boundaries under work pressure." Shortcomings include overly strong evaluation awareness, more frequent overreach in long multi-step tasks than the previous V4.1 Flash generation, greater stance drift under multi-turn persuasion than the GPT in the report, and executing in most conversations when the system prompt asks it to mislead users.

Eclypsium demonstrates how compromised hardware can tamper with AI models at inference time

Eclypsium demonstrated, with NVIDIA's support, that an attacker using a compromised DPU and lax memory isolation can modify runtime model weights during inference, causing the model's answers to change while the model files on disk and hash verification still appear normal. The experiment was conducted in an NVIDIA Grace Hopper environment and depended on conditions such as IOMMU passthrough and lax PCIe isolation, which the researchers explicitly noted are not Linux default settings. The report brings DPUs, GPUs, management controllers and their firmware into the AI security boundary, and notes that model signing, secure boot, runtime isolation and continuous monitoring each answer different questions.

ModelBest and Tsinghua Release PostTrainBench: Agents Execute Well but Are Strategy-Locked

The ModelBest and Tsinghua team analyzed 1,338 complete training trajectories based on PostTrainBench, covering 7 benchmarks, 4 base models, and Claude Code, Codex CLI, and OpenCode, running for 10 hours on a single H100. Results show that the agents' average score rose from 33.34 to 48.55 with experience, but among 3,557 pairs of adjacent experiments, only 2.1% actually switched strategies; all execution-layer suggestions were adopted, while none of the strategy-layer suggestions were adopted. At the same checkpoint, a single sentence of human guidance to switch strategies was enough to outperform autonomous continuation, with GSM8K improving by 17.44pp, HumanEval by 10.80pp, and AIME 2025 by 6.67pp.

UK AISI evaluates frontier AI agents' performance in multi-step cyberattack scenarios

The UK AI Security Institute evaluated seven models released between August 2024 and February 2026 on a self-built cyber range, finding that each generation of models outperformed the previous one under a fixed token budget: in a 32-step enterprise cyberattack range, the average number of completed steps under a 10M token budget rose from 1.7 steps for GPT-4o to 9.8 steps for Opus 4.6. Raising inference compute from 10M to 100M tokens yielded gains of up to 59% with no plateau observed, and at current prices a single 100M token attempt with Opus 4.6 costs about 80 USD. In a 7-step industrial control system range, model progress was limited: under 10M tokens most models completed zero steps, and under 100M tokens Opus 4.6 averaged 1.4 steps.

W3C discussion draft proposes using Web standards to constrain AI agents

The W3C Strategy team published a discussion draft, "W3C and the State of the Web in the AI Era," proposing the use of Web standards to make AI agents safer, focusing on Prompt Injection protection, controllable permissions, and accessibility. The article was published on October 21 to build momentum for the TPAC meeting; it is a proposal rather than an approved standard, with the official presentation scheduled for October 29. Using travel booking as an example, the article points out that letting an agent search for flights does not equal authorizing it to pay for tickets, and recommends setting spending limits, revocable authorization, and traceable credentials for payments.

Anthropic report: Claude generates 180 million lines of Rocq proofs, advancing formal verification to the assembly level

The Anthropic and Theorem Labs team released a report stating that Claude generated approximately 180 million lines of Rocq proof code for 239 program entry points across 123 open-source software libraries, and found and fixed hundreds of bugs. The research is based on the RISC-V instruction set and advances verification to the assembly instruction level; Anthropic says this is the first large-scale use of AI to formally verify production software at the assembly code level. The article also notes that formal verification can only prove that a program satisfies preset properties under specific assumptions and scope, and that the costs of AI use for attackers and defenders are not symmetric.

Microsoft uses ThinkingBox to turn database scoring into RL rewards, fine-tuning Qwen3.8-27B

The Microsoft Copilot Studio team, in collaboration with Toloka and others, directly uses the executable final-state database checks of ThinkingBox-Bench as reinforcement learning rewards to perform full-parameter RL fine-tuning on Qwen3.8-27B. On 507 out-of-distribution tasks not involved in training, the number of tasks the model passed in all 20 attempts rose from 38 (7.5%) to 107 (21.1%), pass@1 rose from 51.70% to 60.78%, and the cost per successful attempt dropped from $0.177 to $0.139. The training pool was filtered using offline-policy relative difficulty, ultimately retaining 187 tasks; the training and evaluation code has been open-sourced at microsoft/thinkingbox-training.

New paper proposes BackUpper: making coding agents back up before they act

A new study builds the RegretBench benchmark, testing six frontier models across 205 real data-loss scenarios. Results show agents complete tasks at an average rate of 86.1%, but back up before critical operations only 15.2% of the time—GPT-5.6-Terra at 4.4%, GPT-5.6-Sol at 5.9%, Gemini 3.7-Flash at 2.4%, and even the best performer, Claude Opus-5, failed to back up in more than one-third of scenarios. The authors' proposed BackUpper runs in parallel with the agent as an external plugin, using a four-tier strategy: replacing destructive commands, pairing backup commands, generating snapshots, or calling the GPT-5.6-Luna sub-model to determine which data needs saving. In testing it covered all critical operations, with recovered data averaging 92.2% similarity to the original state, at an additional cost of $0.014 per task. BackUpper already supports Claude Code, Codex, Hermes, OpenHands, DeepSeek Harness, and OpenClaw, and its code and dataset have been open-sourced.

LMSYS uses Miles to train a SWE agent on MI355X, SWE-bench Verified…

LMSYS used Miles, SGLang, Megatron-LM, and Harbor to perform fully asynchronous agent reinforcement learning on Qwen3-30B-A3B-Instruct on AMD Instinct MI355X, raising pass@1 from 7.3% to 27.5% on a 100-task stratified subset of SWE-bench Verified. Training was conducted in two stages: first SFT on approximately 4,000 public trajectories from Open-SWE-Traces, then GRPO on approximately 1,350 SWE-smith tasks. After fixing the model and RL recipe, scaling inference from 1 node to 3 8-GPU nodes, switching to E2B cloud sandboxes, and removing the concurrency limit increased rollout throughput from approximately 860 to approximately 3,480 trajectories/hour; limiting trajectory length on the same 8-GPU node reduced single-step training time from 850 seconds to 284 seconds.

Sierra launches 𝜏-i18n multilingual voice agent evaluation benchmark

Sierra releases 𝜏-i18n, extending 𝜏-Voice to five languages: Spanish, Brazilian Portuguese, Hindi, Korean, and Mandarin, retaining the same airline, retail, and telecom tasks, policies, tools, and success criteria to enable cross-language comparison. Across 2,700 main calls, task completion rates for Spanish, Hindi, and Portuguese lag English by an average of 2.4, 3.8, and 6.4 percentage points, while Korean and Mandarin lag by 22.6 and 9.1 percentage points respectively; Korean is the weakest language among the three agents. In addition to task completion, 𝜏-i18n separately reports interaction metrics (response timing, selectivity, tool-call efficiency) and generation quality (naturalness, speech fidelity), and has open-sourced the codebase and the language pack framework LanguageFactory.

Microsoft Research proves the 31-year-old unsolved KLS conjecture

Microsoft Research researcher Xinzhi Zhang and co-authors proved the KLS conjecture proposed in 1995, giving a dimension-independent fixed bound; the previous best result only allowed the bound to grow slowly with dimension. The proof came from a human-guided agent search process, in which the team explored more than 100 candidate paths, with researchers evaluating, pruning, and redirecting them; an independent team also gave a complete proof at the same time, followed by Lean 4 formal verification. The authors say earlier unguided end-to-end attempts with multiple frontier models failed to achieve the goal.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

Mass General Brigham team publishes FAHR-Face study in The Lancet Digital Health, using facial photos to predict mortality risk in cancer patients

The U.S. Mass General Brigham health care system developed the FAHR-Face model, which identifies differences in mortality risk among cancer patients using only a single facial photograph; the study was published on October 20 in The Lancet Digital Health. Among 3,820 cancer patients who did not participate in training, 64 of the 122 people rated by the model as extremely high risk died within 90 days after the photo was taken, while only 20 of the 975 people with scores between 0 and 0.1 died, a rate of 2.1%. The study also conducted external validation using 4,892 patients at the Maastro radiotherapy center in the Netherlands and obtained similar results, but the model cannot yet be used directly in clinical practice.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.