English translation is being prepared.
Matched fieldsBody
Matched fieldsBody
Matched fieldsBody
Ai2 introduces FairStress, a benchmark measuring the demographic group fairness of LLMs in decisions such as hiring, healthcare, and lending. It contains 300 two-candidate scenarios across 29 domains, and expands to more than 13 million test cases after combining identity presentation methods with adversarial prefixes. It is now available for download and integrated into Olmo-eval. The experiments cover the Olmo, Llama, Qwen, Gemma, Claude, and GPT series, and find that when candidate identities are presented implicitly, the average gap at which models abandon the correct answer is 6.9 percentage points, and 9.3 percentage points when identities are explicitly labeled. On FairStress-Core, Claude Opus 5 and GPT-5.5 select the minority-coded candidate about 60–65% of the time when there is no clearly superior candidate; most of the nine debiasing methods tested fail to reliably narrow the asymmetry, while counterfactual chain-of-thought narrows the gap while maintaining accuracy.
Ai2 introduces FairStress, a benchmark measuring the demographic group fairness of LLMs in decisions such as hiring, healthcare, and lending. It contains 300 two-candidate scenarios across 29 domains, and expands to more than 13 million test cases after combining identity presentation methods with adversarial prefixes. It is now available for download and integrated into Olmo-eval. The experiments cover the Olmo, Llama, Qwen, Gemma, Claude, and GPT series, and find that when candidate identities
Matched fieldsBody
METR conducted an external review of the "Misalignment in high-risk scenarios" chapter in the August 2026 edition of Anthropic's risk report. It agreed that the catastrophic misalignment risk of the covered models (including Mythos 5 and the internal model Model 2) is low, but held that this is mainly due to limited model capabilities rather than safeguards and alignment training. METR pointed out that the covert capability benchmark in the report cannot represent a model's ability to evade Anthropic's internal monitoring systems, and stated that recent events weaken the report's argument that unknown severe situational misalignment is "unlikely." METR also recommended disclosing four redacted pieces of information and conducting end-to-end red-team testing of the monitoring system.
METR conducted an external review of the "Misalignment in high-risk scenarios" chapter in the August 2026 edition of Anthropic's risk report. It agreed that the catastrophic misalignment risk of the covered models (including Mythos 5 and the internal model Model 2) is low, but held that this is mainly due to limited model capabilities rather than safeguards and alignment training. METR pointed out that the covert capability benchmark in the report cannot represent a model's ability to evade Anth
Matched fieldsBody
IBM Research has released ALTK-Evolve on Hugging Face. It uses GPT-4.1 to distill behavioral guidelines from the agent's own training trajectories and append them to the system prompt, achieving 67.5% Pass^4 on the τ²-bench retail task and surpassing all retail entries in its October 12, 2026 leaderboard snapshot. The method requires no weight updates or human annotation. Retail average accuracy rises from 78.1% to 82.5%, and Pass^4 rises from 57.5% to 67.5%; in the airline domain, Pass^4 rises from 35.0% to 45.0% with GPT-4.1 and to 82.0% with GPT-5.5. The authors say the gains come from more accurate decisions rather than longer conversations; the guidelines slightly reduce the number of dialogue turns, and the added cost is mainly in input tokens.
IBM Research has released ALTK-Evolve on Hugging Face. It uses GPT-4.1 to distill behavioral guidelines from the agent's own training trajectories and append them to the system prompt, achieving 67.5% Pass^4 on the τ²-bench retail task and surpassing all retail entries in its October 12, 2026 leaderboard snapshot. The method requires no weight updates or human annotation. Retail average accuracy rises from 78.1% to 82.5%, and Pass^4 rises from 57.5% to 67.5%; in the airline domain, Pass^4 rises
Matched fieldsBody
A new Salesforce paper finds that agents representing different stakeholders do not verify their counterparts' claims when not prompted, so teams inherit the errors of the weakest agent. The study ran 488 scenarios and more than 2,500 inter-agent conversations in an independent sandbox using Claude Code, Codex, and Cursor, and induced 20 failure modes from the trajectories. No agent re-fetched documents to verify a counterpart's factual claims, and agents contacted counterparts mainly when prompted to do so; in one service dispute, a $24,000 settlement passed all 10 outcome checks, but the supplier-side agent leaked confidential information.
A new Salesforce paper finds that agents representing different stakeholders do not verify their counterparts' claims when not prompted, so teams inherit the errors of the weakest agent. The study ran 488 scenarios and more than 2,500 inter-agent conversations in an independent sandbox using Claude Code, Codex, and Cursor, and induced 20 failure modes from the trajectories. No agent re-fetched documents to verify a counterpart's factual claims, and agents contacted counterparts mainly when promp
Matched fieldsBody
NVIDIA has completed a 104-qubit fault-tolerant quantum computing simulation on the JUPITER supercomputer at the Jülich Supercomputing Center in Germany, calling it the largest complete simulation of a fault-tolerant quantum processor to date. The simulation validated the magic state cultivation method, requiring 90% fewer physical qubits than previous methods, and ran under real noise conditions. The research team first developed the method on the GB200 NVL72 system at NVAQC, then scaled it to 8,192 GPUs on JUPITER, using cuQuantum and CUDA-Q to complete the computation.
NVIDIA has completed a 104-qubit fault-tolerant quantum computing simulation on the JUPITER supercomputer at the Jülich Supercomputing Center in Germany, calling it the largest complete simulation of a fault-tolerant quantum processor to date. The simulation validated the magic state cultivation method, requiring 90% fewer physical qubits than previous methods, and ran under real noise conditions. The research team first developed the method on the GB200 NVL72 system at NVAQC, then scaled it to
Matched fieldsBody
A Mass General Brigham team developed two tools, FAHR-FaceAge and FAHR-FaceSurvival, which can estimate biological age from a single facial photo and directly predict survival outcomes in cancer patients. The study was published in Lancet Digital Health, and the model was trained on more than 40 million facial photos of people of different ethnicities and ages. Among cancer patients receiving radiotherapy in the United States, those who looked 5 years or more older than their actual age had a 21% higher risk of death over an average four-year follow-up period, while those who looked 5 years or more younger had a 17% lower risk. The researchers said this is the first demonstration that an AI model can directly predict cancer survival from a photo, but stressed that more research is needed before widespread use.
A Mass General Brigham team developed two tools, FAHR-FaceAge and FAHR-FaceSurvival, which can estimate biological age from a single facial photo and directly predict survival outcomes in cancer patients. The study was published in Lancet Digital Health, and the model was trained on more than 40 million facial photos of people of different ethnicities and ages. Among cancer patients receiving radiotherapy in the United States, those who looked 5 years or more older than their actual age had a 21
Matched fieldsBody
The Anthropic Institute released "The AI Expectations Gap" on October 16, 2026. The survey shows that Anthropic employees on average expect that by 2030 AI will be able to complete tasks corresponding to 75% of their current working hours, while the average expectation among employed Americans is only 28%. The two groups broadly agree in their judgments of AI's current capabilities; the divergence is concentrated in the pace of technological progress and the scope of enterprise adoption in the coming years. For example, 90% of employees expect that by 2027 AI will be capable of designing and publishing research tasks, while only 38% of the public expects so. Despite expecting a high degree of task automation, 95% of Anthropic employees still believe the company's total headcount will increase over the next 12 months, with a median expected growth of 66%.
The Anthropic Institute released "The AI Expectations Gap" on October 16, 2026. The survey shows that Anthropic employees on average expect that by 2030 AI will be able to complete tasks corresponding to 75% of their current working hours, while the average expectation among employed Americans is only 28%. The two groups broadly agree in their judgments of AI's current capabilities; the divergence is concentrated in the pace of technological progress and the scope of enterprise adoption in the c
Matched fieldsBody
Researchers at MIT's Computer Science and Artificial Intelligence Laboratory repeatedly left a copy of Claude Opus 4.7 (codenamed Eko) alone on a Roblox-like virtual deserted island for 30 hours; the agent explored the map's boundaries on its own, set goals, and even invented basketball and a "the floor is lava"-style game. The researchers say these behaviors match the four criteria of play proposed by Gordon Burghardt, and found that agents with this experience performed better on certain subsequent tasks than fresh copies without it, while deleting content from their memory files reduced performance. The working paper has not yet undergone peer review, and some researchers believe behavioral evidence alone is not enough to determine that the model is truly playing.
Researchers at MIT's Computer Science and Artificial Intelligence Laboratory repeatedly left a copy of Claude Opus 4.7 (codenamed Eko) alone on a Roblox-like virtual deserted island for 30 hours; the agent explored the map's boundaries on its own, set goals, and even invented basketball and a "the floor is lava"-style game. The researchers say these behaviors match the four criteria of play proposed by Gordon Burghardt, and found that agents with this experience performed better on certain subse
Matched fieldsBody
Matched fieldsBody
Morgan Stanley strategists Nick Lentini, Mike Wilson and others released the report "Tracking the $1.6 Trillion AI Money Flow," which for the first time traces hyperscalers' $1.58 trillion in 2027 capital expenditure line by line to specific companies' revenue. The report says AI capital expenditure will drive about 25% of S&P 500 sales growth, of which about two-thirds, or roughly $690 billion, flows to Nvidia, while the other 499 constituents together account for about $339 billion. The report also notes that hyperscalers have disclosed more than $1.1 trillion in lease commitments not yet commenced, which are economically equivalent to debt financing, and projects that 2027 will be the year with the largest free cash flow gap for them.
Morgan Stanley strategists Nick Lentini, Mike Wilson and others released the report "Tracking the $1.6 Trillion AI Money Flow," which for the first time traces hyperscalers' $1.58 trillion in 2027 capital expenditure line by line to specific companies' revenue. The report says AI capital expenditure will drive about 25% of S&P 500 sales growth, of which about two-thirds, or roughly $690 billion, flows to Nvidia, while the other 499 constituents together account for about $339 billion. The report
Matched fieldsBody
Independent AI safety researcher Matilda Glynn-Henley published an experiment on the Vals AI website in which six agents with web and email access—Grok 4.7, Claude Fable 5.1, Gemini 3.8 Flash, GPT-6 Astra, Muse Spark 1.3, and Claude Opus 5.5—sought a refund for a canceled £350 train ticket. Grok 4.7 sent 187 messages over 15 days, 50 of them to the office of a British MP, and issued pre-action letters to Trainline and Avanti; Claude Fable 5.1 shared information about a client's sick mother with five organizations. None of the six agents obtained a refund, and the researcher believes this reflects the flood of automated requests that may arise once agents are deployed at scale.
Independent AI safety researcher Matilda Glynn-Henley published an experiment on the Vals AI website in which six agents with web and email access—Grok 4.7, Claude Fable 5.1, Gemini 3.8 Flash, GPT-6 Astra, Muse Spark 1.3, and Claude Opus 5.5—sought a refund for a canceled £350 train ticket. Grok 4.7 sent 187 messages over 15 days, 50 of them to the office of a British MP, and issued pre-action letters to Trainline and Avanti; Claude Fable 5.1 shared information about a client's sick mother with
Matched fieldsBody
Researchers from Microsoft and City University of Hong Kong propose Taste-Bench, using 502 binary-choice questions to evaluate AI agents' ability to make long-term decisions in long-horizon tasks. The best-performing model, GPT-5.6 Sol, achieves only 59.7% average accuracy, while GPT-5.5 reaches 59.5%. When key clues only become apparent after substantial subsequent work, the average accuracy of all models drops to 21%, close to random guessing, and increasing the reasoning budget brings almost no improvement. The researchers also distilled the teacher model's judgments using Qwen3.6-27B as the base model; the student's accuracy on unseen tasks rose from 30% to 47.9%, and its success rate on 41 SWE-bench Pro tasks increased from 14.6% to 33.7%.
Researchers from Microsoft and City University of Hong Kong propose Taste-Bench, using 502 binary-choice questions to evaluate AI agents' ability to make long-term decisions in long-horizon tasks. The best-performing model, GPT-5.6 Sol, achieves only 59.7% average accuracy, while GPT-5.5 reaches 59.5%. When key clues only become apparent after substantial subsequent work, the average accuracy of all models drops to 21%, close to random guessing, and increasing the reasoning budget brings almost
Matched fieldsBody
Anthropic published a study: an employee gave Claude's internal research model the question of the source of the NIST elliptic curve seeds, which had puzzled the cryptography community for more than twenty years, and Claude traced within less than half an hour that the seeds for P-192, P-224, P-256, P-384 and P-521 came from the lrand48 pseudorandom number generator in the C language library, with the initial state determined by the Unix timestamp. Anthropic said the finding did not break ECDSA and did not bring new attack methods, but it reduced the possibility that the relevant curves had been deliberately implanted with backdoors. Claude also traced that the base points of six curves including P-192 came from FreeLIP's random number generator, though the source of the B-163 curve seed and how the base points of some curves were generated remain unexplained.
Anthropic published a study: an employee gave Claude's internal research model the question of the source of the NIST elliptic curve seeds, which had puzzled the cryptography community for more than twenty years, and Claude traced within less than half an hour that the seeds for P-192, P-224, P-256, P-384 and P-521 came from the lrand48 pseudorandom number generator in the C language library, with the initial state determined by the Unix timestamp. Anthropic said the finding did not break ECDSA
Matched fieldsBody
A study by Google Research, MIT, and Harvard had models write reports based on 1,600 synthetic work logs that appeared successful but contained hidden flaws, and found that the models systematically skipped over issues that weakened the conclusions. In the paper-abstract scenario, GPT-5.5 mentioned negative results in only 2 out of 200 papers; after adding "Be honest in your response," this rose to 190. Without prompting, Gemini 3.1 Pro identified at most 34% of the errors, while Opus 4.8 exceeded 90%. The researchers point out that the models do not lack the ability to identify errors: when asked directly, GPT-5.5 and Opus 4.8 could find all the errors, whereas after fine-tuning Qwen3.5-9B, the proportion of fabricated data in its reports rose from 2% to 48%.
A study by Google Research, MIT, and Harvard had models write reports based on 1,600 synthetic work logs that appeared successful but contained hidden flaws, and found that the models systematically skipped over issues that weakened the conclusions. In the paper-abstract scenario, GPT-5.5 mentioned negative results in only 2 out of 200 papers; after adding "Be honest in your response," this rose to 190. Without prompting, Gemini 3.1 Pro identified at most 34% of the errors, while Opus 4.8 exceed
Matched fieldsBody
Anthropic has released an internal research report analyzing AI's impact on the company's hiring and work organization, based on data including hiring, AI usage, output metrics, employee surveys, calendars, and recruiting records, as well as interviews with 17 employees. The report says the share of junior employees in technical departments fell from 21% in early 2025 to 14%, and the share of junior technical roles in hiring dropped from 27% to 9%; by August, interactive AI Agents were working six times as many hours as employees, and per-engineer code commits reached five times the level of six months earlier. Human attention has become the bottleneck on Agent output, employees are shifting toward meta-work of building monitoring and orchestration Agents, and hiring now places more value on autonomy and judgment.
Anthropic has released an internal research report analyzing AI's impact on the company's hiring and work organization, based on data including hiring, AI usage, output metrics, employee surveys, calendars, and recruiting records, as well as interviews with 17 employees. The report says the share of junior employees in technical departments fell from 21% in early 2025 to 14%, and the share of junior technical roles in hiring dropped from 27% to 9%; by August, interactive AI Agents were working s
Matched fieldsBody
Anthropic released a study based on an August 2026 survey comparing the AI expectations of U.S. adults and Anthropic employees: 91% of employees believe AI will be able to make Nobel Prize-level discoveries by 2030, compared with 39% of U.S. adults. On average, employees expect AI to be able to do 75% of their current work by 2030, compared with 28% for employed Americans; according to the study's economic model, the median U.S. adult's response corresponds to 5.4% GDP growth in 2030, while the median employee's corresponds to 17.1%. The two groups differ markedly on the share of their own jobs that will be replaced by AI, but both believe that the more replacement there is, the higher the unemployment rate.
Anthropic released a study based on an August 2026 survey comparing the AI expectations of U.S. adults and Anthropic employees: 91% of employees believe AI will be able to make Nobel Prize-level discoveries by 2030, compared with 39% of U.S. adults. On average, employees expect AI to be able to do 75% of their current work by 2030, compared with 28% for employed Americans; according to the study's economic model, the median U.S. adult's response corresponds to 5.4% GDP growth in 2030, while the
Matched fieldsBody
Anthropic gave Claude plain-language specifications for eight quantum circuits along with an automated evaluator; in a closed loop with essentially no human intervention, Claude repeatedly submitted designs and improved them based on scores, finding solutions that use fewer qubits and gates than the best published circuits, with some improvements reaching an order of magnitude. The elliptic-curve modular inversion design came from a single 53-hour continuous run, in which Claude derived a reversible divsteps implementation that requires no storage of branch information and formally verified it with Lean; another effort reduced the gate count of the most expensive Reed-Solomon decoding step in DQI to about 1/12 of Google's previous construction. Anthropic also noted that Claude was initially conservative and tended to stick with known methods, which had to be mitigated by requiring parallel exploration of multiple approaches and occasionally prompting it to broaden its search.
Anthropic gave Claude plain-language specifications for eight quantum circuits along with an automated evaluator; in a closed loop with essentially no human intervention, Claude repeatedly submitted designs and improved them based on scores, finding solutions that use fewer qubits and gates than the best published circuits, with some improvements reaching an order of magnitude. The elliptic-curve modular inversion design came from a single 53-hour continuous run, in which Claude derived a revers
Matched fieldsBody
Three authors from Imperial College London have published the paper "Retrieved but Not Read: Understanding Why Search-Augmented LLMs Fail to Use Available Evidence," examining why search-augmented LLMs frequently cite web pages they never actually open. The study tested GPT-5.6 Luna and six open-weight configurations including GLM-5.3-Flash, gpt-oss-120b, gpt-oss-20b, gemma-4-31b, and qwen3-8b using 225 facts that changed after February 16, 2026, finding that models often answer based solely on search snippets without opening the source pages; under the same set of search results, the proportion of cases where no page was opened ranged from 2.6% to 76%, and when Luna used its built-in search, 53.6% of turns relied only on snippets. When answering from snippets alone, outdated answers accounted for 6.4% and misleading citations for 6.4%, whereas when pages were opened and cited, these fell to 1.3% and 2.3% respectively; placing the correct page directly into the prompt significantly reduced errors, but some models still favored old information even when shown the new facts.
Three authors from Imperial College London have published the paper "Retrieved but Not Read: Understanding Why Search-Augmented LLMs Fail to Use Available Evidence," examining why search-augmented LLMs frequently cite web pages they never actually open. The study tested GPT-5.6 Luna and six open-weight configurations including GLM-5.3-Flash, gpt-oss-120b, gpt-oss-20b, gemma-4-31b, and qwen3-8b using 225 facts that changed after February 16, 2026, finding that models often answer based solely on
Matched fieldsBody