Gemini 4 Carbon slightly trails Claude on the GDPx benchmark, but offers better cost-effectiveness
SemiAnalysis has released GDPx, a frontier model benchmark covering 206 tasks across finance, materials science, cybersecurity, software engineering, law, biology, and other fields, and it lists both the scores and costs of each model. According to a 9to5Google report, Google's Gemini 4 Carbon slightly trails Claude on the leaderboard but performs better per unit cost; if cybersecurity, biology, and alignment are excluded, GPT-6 Astra ranks first, and GPT 6.1 Sol and Claude Sonnet 5.5 also rank ahead of Gemini 4 Carbon. Google's existing model Gemini 4 Argon ranks eighth on both leaderboards, with a cost only slightly lower than Carbon; Google released Gemini 4 Carbon on October 28, and most customers still need to wait for it to become available.
Why read this
AI score 60
SemiAnalysis's GDPx benchmark lists both scores and costs, allowing readers to compare the cost-effectiveness differences among Gemini 4 Carbon, Claude, and GPT.
Topics
Body
Google’s new frontier model, Gemini 4 Carbon, brings some welcome improvements and while it doesn’t top the charts in a new benchmark, it does provide much more bang for your buck.
SemiAnalysis, an AI research and analysis firm, recently published its new frontier-level benchmark, GDPx which includes numbers for Google’s Gemini 4 Carbon model. The leaderboard includes every major frontier model from the likes of OpenAI, Anthropic, and Meta and details both the score and cost for each model.
The benchmark includes 206 tasks across finance, materials science, cybersecurity, software engineering, legal, biology, and several other domains (topics).
The report notes that taking out cybersecurity, biology, and alignment domains shuffles the results. Safety guardrails in Claude and GPT models puts GPT-6 Astra at the top of the list with GPT 6.1 Sol and Claude Sonnet 5.5 outranking Gemini 4 Carbon. SemiAnalysis says that this better aligns with their “subjective impressions” of each model.
As for Google’s existing frontier model, Gemini 4 Argon, it ranks in eighth place on both lists, beating Meta’s Muse Wonder, but below Wonder Full. Argon’s costs in this particular benchmark are only slightly lower than Carbon.
Both of Google’s models score better than OpenAI and Anthropic models when focused on cybersecurity, but do fall behind Meta’s Muse models, which the benchmark found to be the best “by a meaningful margin.”
Google announced Gemini 4 Carbon on October 28 and touted “significant gains,” but the public release date has not been confirmed just yet. It is “coming soon” for most customers. Google’s own benchmark, below, beat GPT-6 Astra and Opus 5.5 in many aspects.
More on Gemini:
Latest Gemini design comes to all Android devices, app upgrades file generation
Google adding more Connected Apps to Gemini & Search with new dev platform
Google says Gemini 4 Argon ranks well behind Claude and GPT-6 for Android coding