Sundar Pichai praises Carbon's benchmark quality and cost performance

Nov 2

1. 13:35

   Unite.AI

   SemiAnalysis releases the GDPx benchmark, covering 11 agent domains

   SemiAnalysis releases the GDPx benchmark, covering 11 professional domains including chip design, cybersecurity, biology, finance, and law. The first version contains 206 tasks built by 14 data partners, and the leaderboard includes 13 models. Claude Opus 5.5 ranks first at 12.0%, with an average cost of $51.41 per task and 548K output tokens; Gemini 4 Carbon ranks second at 10.9%, and GPT-6 Astra ranks third at 9.6%. SemiAnalysis says the benchmark is updated every 3 to 6 months, and criticizes OpenAI GDPval's tasks as non-interactive, with context written directly into the prompt.

   Expand summary
2. 12:34

   Sundar Pichai

   Pichai comments on the SemiAnalysis GDPx benchmark and Carbon costs

   SemiAnalysis released the GDPx benchmark, used to measure the performance of agents in high-value domains, saying that the strongest model scored only 12%. Sundar Pichai reposted it and commented that he was glad to see a high-quality benchmark with room for improvement, and specifically mentioned Carbon's cost curve.

   Expand summary