Mollick reviews SemiAnalysis's new benchmark GDPx
SemiAnalysis released the GDPx benchmark to measure agents' capabilities in high-value domains, and the highest-scoring model scored only 12%. Ethan Mollick considers it a worthy successor to GDPval, but notes that it lacks a human baseline like GDPval has; he also mentioned that GPT-6 Pro found some errors in the scoring, and believes that the 0% scores obscure economically meaningful differences in success rates.
View original postToday we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵 https://t.co/RNnkcuUkih
View original
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵 https://t.co/RNnkcuUkih
Body
Nice to see a successor to GDPval to evaluate AI work but really needs human baselines like GDPval.
Funnily enough, GPT-6 Pro spotted some errors in the grading. I also think the 0% scores hide economically meaningful differences in success rates. But still a valuable benchmark! https://t.co/mfKfsKzcpM