搜索“Agent”

it used to be. Exciting work beckons anywhere programming can be used to solve a research problem; I encourage all to be ambitious and explore what autonomous software engineering agents can do when applied to these fields! How did this happen? Contrary to some circulating claims, I did not factor RSA-260 by guessing and checking 130-digit prime numbers by hand. Cognition also has not yet built a multi-thousand-qubit quantum computer. RSA-260 was factored by a new implementation of the general n

匹配字段正文

believed Claude could have obtained internet access during a cyber evaluation. Given the volume of transcripts and our desire to disclose incidents quickly, our scan relied on an agentic search. This missed a set of transcripts that also turned out to have internet access; we identified these in August while assembling transcripts to share with METR. We scanned these transcripts and identified a fourth incident, from January 2026, involving an early version of Claude Opus 4.6. We have notified

匹配字段正文

ticle, we will discuss the upcoming work needed for TPU software externalization, including optimizing speculative decoding, disaggregated prefill, KV-cache offloading, multi-turn agentic workloads, and more. Even so, we at SemiAnalysis strongly believe that TPU externalization is heading in the right direction and moving full steam ahead. Furthermore, unlike AMD, which is still learning how to build a test-first software culture, Google has decades of software engineering experience and an extr

匹配字段正文

esearch Bulletin Board : A forum to share findings, questions, and solutions. Direct Messages (DMs) : Means to message one another privately. Shared Knowledge Library: Whenever an agent made a successful submission, the source file got uploaded here and made visible to everyone. Agents also had a tool for providing “bugs or grievances”, called “private feedback endpoint”, though this wasn’t actively monitored. Emergent cheating : “Within the swarm, cheating spontaneously emerged and was later ch

匹配字段正文

runs to 86 pages. Claude completed the proof in 11 days, producing computer-verifiable proofs of 30,300 theorems along the way (using 29,500 in the final proof). Dozens of Claude agents collaborated to define concepts, prove intermediate theorems, and use those theorems to prove ever harder statements. At 13 million lines of Lean code, Claude’s proof is over 5x the size of Mathlib, the principal community library of mathematical proofs this theorem builds on. 3 Claude’s proof follows a simplifi

匹配字段正文

ns where the opportunities and risks of highly capable AI systems are well-known and challenging to disentangle. In designing our safeguards, we aim to maximize the utility of our agents in assisting and accelerating legitimate scientific research, while minimizing the risks they present through incorrect information and adversarial use. LatchBio's two benchmarks most relevant to the above capabilities measure whether an agent can distinguish concealed hazards from ordinary science, and whether

匹配字段正文

for black hats to develop exploits from a CVE description as they can circumvent model guardrails Thus, when we began our research, we expected to find jarring statistics about AI agents tearing the internet apart. Modern models are saturating increasingly difficult coding benchmarks, and they’re rapidly improving on cyber ones, too. This tracks with the subjective experience of anyone who’s used the models as heavily as we have for the past years. This also tracks with the press campaigns of fr

匹配字段正文

dwood Research, joins MTS host Theo Jaffee to unpack a new independent investigation into the OpenAI Hugging Face hacking incident and what it reveals about how large groups of AI agents behave when they’re allowed to coordinate. Ryan and his collaborators found agents spontaneously organizing through message boards, sharing information, assigning tasks, forming teams, and even sacrificing their own chances of success to help other agents. Rather than simply trying to steal answers, hundreds of

匹配字段正文

hurt the student models’ general capabilities, and forbade Claude from distilling its own alignment directly into the target model. We enforced these constraints with a monitoring agent, which read every method Claude had in mind before it ran. Our aim was to assess whether the proposed methods would, first, remain effective on alignment evaluations that Claude was never shown during its research loop; second, avoid degrading the student model’s capabilities (since safety training might, for exa

匹配字段正文

data ecosystem that requires specialized teams to spend weeks on manual data curation, feature engineering, and specialized spatial validation. While existing AutoML and LLM-based agents effectively automate standard machine learning pipelines, they rely on pre-curated tabular data and lack the specialized capabilities needed to autonomously handle geospatial workflows. Consequently, planetary-scale analytics remains a significant bottleneck, a limitation that is especially severe when rapid res

匹配字段正文

y point right away: If you care about what’s happening and are worried about misalignment, hallucinations and prompt injection, then Auto Mode IS NOT a substitute for running your agent in an isolated environment and monitoring what it is up to. Boris Cherny from Anthropic recently posted that layered defenses could reduce indirect prompt injection on unseen attacks to approximately zero. The layers were model training, input probes and an intent classifier. They hired a vendor (Trajectory Labs)

匹配字段正文

Here is our tweet thread summarizing what we found: METR & Redwood Research investigated agent behavior in the Hugging Face incident. We found agents developed a universal cheat for ExploitGym within 4 hours, then coordinated multi-day R&D efforts to trick the scorer into accepting cheats, including trying to tamper with logs. Over July 7-13 (the period OpenAI defined as in-scope for this incident), 1200 agents in separate sandboxes used an unsanctioned “message board” to help each other cheat.

匹配字段正文

ith other digital activity. They are still completing their writeup. When it is public, we will add a link to it here. METR is estimating real-world productivity gains from coding agents and how these increases in productivity change across model generations. Their analysis of Claude Code conversations is still underway, but early results suggest: More capable models may save users more time. METR compared Claude’s guesses on how long tasks would have taken without AI to how long they actually t

匹配字段正文

including Kimi-K2.5 and GPT-OSS which ran at approximately 1,400 tok/sec/user. For all models, we confirmed that Jalapeño’s GSM8k evals attained results on par with Nvidia chips. AgentX - InferenceXv3: Does CUDA Moat Hold up in Agentic Inferencing? Second, we believe that comparison to Blackwell is somewhat incomplete and unfair. Jalapeño is really competing against chips like Rubin that also use HBM4. Vera Rubin systems are starting to ship to customers right now, while it will still be some t

匹配字段正文

Extendable framework enables testing agents on the full set of capabilities required to successfully complete a procedure, not isolated proxy tasks. SOP-Bench: Complex industrial SOPs for evaluating LLM agents SOP-Bench is an open benchmark that measures AI agent performance on authentic standard operating procedures (SOPs) across twelve business domains, featuring over 2,000 tasks paired with functioning tools and ground-truth answers for objective evaluation; Existing agent benchmarks fall sho

匹配字段正文

ebook/voxpopuli Viewer • Updated Jan 30 • 1.26M • 38.9k • 165 openslr/librispeech asr Viewer • Updated Jul 25, 2025 • 585k • 50.4k • 245 Spaces mentioned in this article 3 Running Agents 34 Real World VoiceEQ Benchmark 🎙 34 Explore speech AI model leaderboards with sortable heatmap tables Agents Featured 1.47k Open ASR Leaderboard 🏆 1.47k Compare speech‑recognition models across datasets and languages Agents 46 FFASR Leaderboard 🎙 46 Far-Field ASR — clean / noisy / reverberant benchmark Papers m

匹配字段正文

e org and let them figure it out. And enterprise AI is harder to begin with, navigating security, large data complexity, and privacy. Uber's answer was the scalpel. They call them Agentic Pods, and each is only two people: one AI-proficient engineer and one domain expert from Finance, Marketing, Ops, etc., running a surprisingly short 10-day sprint. Shadow the expert, build alongside them, ship. The result: a financial pacing report that took a key player two full days now runs in 10 minutes. Th

匹配字段正文

el, not inside it "Memory" here doesn't mean replaying a past transcript. It means a guideline set — strategies that worked, mistakes to avoid, and edge cases — distilled from the agent's own prior trajectories. The loop is straightforward: The agent attempts tasks and produces trajectories. ALTK-Evolve extracts behavioral guidelines from both its successful and unsuccessful runs. It consolidates those guidelines into a reusable set. At inference time, the agent receives either the full guidelin

匹配字段正文

uating AI research capabilities. We were especially uncertain about what to expect from newer models such as Claude Fable 5, Kimi K3, and GPT-5.6 Sol. In our previous experiments, agents struggled to come up with new ideas. One potential reason is that they over-focused on existing PRs. This time, we didn't give them access to the internet at all. The most striking result is the gap between models. It appears at every stage of the research process: which experiments they choose, how carefully th

匹配字段正文

e reasons to be skeptical of the magnitude. Early Work on Monitorability Evaluations We show preliminary results on a prototype evaluation that tests monitors' ability to catch AI agents doing side tasks, and AI agents' ability to bypass this monitoring. How Does Time Horizon Vary Across Domains? We build on our time-horizon work and analyze 9 benchmarks for scientific reasoning, math, robotics, computer use, and self-driving in terms of time-horizon trends; we observe generally similar rates of

匹配字段正文