SemiAnalysis releases the GDPx benchmark, testing agents on expert work across 11 domains
SemiAnalysis releases the GDPx benchmark, testing AI agents with expert tasks across 11 domains; the top-scoring Claude Opus 5.5 averages only 12%, followed by Gemini 4 Carbon at 10.9%. The tasks span chip design, cybersecurity, finance, and law, among others, and come from 14 specialized data partners; unlike OpenAI's GDPval, which provides context directly in the prompt, GDPx requires agents to find information themselves in workspaces such as a replica of a real acquisition case made up of 3,983 documents. On financial and legal tasks, all models scored 0 under pass/fail grading, so partial credit was used instead; of 111 attempts to improve existing trading strategies, 70 saw the model perform worse than the original strategy, often simply by increasing risk.
Why read this
AI score 69
GDPx feeds context through realistic workspaces rather than prompts, and all models score zero on financial and legal tasks, revealing the gap in agents' ability to do expert work.
Topics
Body
SemiAnalysis has released GDPx, a benchmark testing AI agents on expert work across 11 domains.
The top model, Anthropic’s Claude Opus 5.5, averages just 12%, with Google’s Gemini 4 Carbon next at 10.9%.
OpenAI’s GPT-6 Astra tops AI-research and materials-science tasks.
The tasks, spanning chip design, cybersecurity, finance and law among others, come from 14 specialized data partners.
Unlike OpenAI’s GDPval, which feeds models context in the prompt, GDPx makes agents dig through workspaces such as a 3,983-file replica of a real acquisition.
Excluding cyber, biology and alignment, where safety classifiers sometimes intervene, lifts OpenAI’s GPT-6 Astra from 9.6% to 11.8%, past Opus.
Finance and law defeat every model: all score 0 under pass-fail grading, so GDPx reports partial credit instead.
Asked to improve a working trading strategy, models did worse than the original in 70 of 111 attempts, often just taking bigger risks instead of finding better trades.