←Search results

Search “DeepMind”

By relevance · Total 18 results Clear search

Thursday

  1. ArenaSelected

    Gemini 4 Carbon ranks 2nd on Agent Arena with a net improvement of 16.2%

    Arena.ai says Gemini 4 Carbon ranks 2nd in the real-task-based Agent Arena with a net improvement of 16.2%, and sits on the Pareto frontier for cost, time, and output tokens per task. Compared with Gemini 4 Argon, which ranks 7th with an 11% improvement, Carbon is 5 places higher and 5.2 percentage points better; the gains mainly come from a +13.2 percentage point increase in the positive-feedback-to-complaint ratio, +5.6 percentage points in confirmed success, and +4.8 percentage points in steerability. Arena says Carbon's availability will be announced by Google DeepMind.

    a net improvement of 16.2%, and sits on the Pareto frontier for cost, time, and output tokens per task. Compared with Gemini 4 Argon, which ranks 7th with an 11% improvement, Carbon is 5 places higher and 5.2 percentage points better; the gains mainly come from a +13.2 percentage point increase in the positive-feedback-to-complaint ratio, +5.6 percentage points in confirmed success, and +4.8 percentage points in steerability. Arena says Carbon's availability will be announced by Google DeepMind.

    Matched fieldsSummary、Body

    #editorial#Evaluations / benchmarks#Agents#Google
  2. Artificial AnalysisSelected

    Gemini 4 Carbon Tops the Artificial Analysis Intelligence Index…

    Artificial Analysis evaluations show that Google's Gemini 4 Carbon tops Intelligence Index v5 with 59 points, ranking first in four of the 9 evaluations: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience, and AutomationBench-AA. On the Coding Agent Index, Gemini 4 Carbon (high) paired with Antigravity CLI scores 59 points, 3 points behind Claude Code paired with Claude Opus 5.5 (max, 62 points). The model is currently available only to some users and has not yet been publicly released; it is priced at $4/$20 per 1M input/output tokens with a 50% discount, has a 1M-token context window, and supports text, image, video, and voice input.

    Artificial Analysis evaluations show that Google's Gemini 4 Carbon tops Intelligence Index v5 with 59 points, ranking first in four of the 9 evaluations: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience, and AutomationBench-AA. On the Coding Agent Index, Gemini 4 Carbon (high) paired with Antigravity CLI scores 59 points, 3 points behind Claude Code paired with Claude Opus 5.5 (max, 62 points). The model is currently available only to some users and has not yet been publicly released; it is p

    Matched fieldsBody

    #editorial#Evaluations / benchmarks#Reasoning#English text is being prepared.

Friday

  1. The Decoder

    Google Gemini 4 Argon scores 97.5% on ARC-AGI-3, but drops to 49.8% when the test environment is changed

    Google DeepMind's new frontier model Gemini 4 Argon scored 97.5% on ARC-AGI-3 in the Provider Adapter Harness test environment at a cost of $4,800, becoming the second lab after OpenAI to cross 95%, second only to GPT-6 Astra's 99.9%. Under the Standard Harness, which allows only simple notes, the low-reasoning setting's score dropped to 49.8%, and due to API issues it was unable to complete all 55 games. On Epoch AI's ECI index, Argon scored 165, on par with Fable 5.1, but ranked eighth on the math ECI and sixth on the software engineering index.

    Google DeepMind's new frontier model Gemini 4 Argon scored 97.5% on ARC-AGI-3 in the Provider Adapter Harness test environment at a cost of $4,800, becoming the second lab after OpenAI to cross 95%, second only to GPT-6 Astra's 99.9%. Under the Standard Harness, which allows only simple notes, the low-reasoning setting's score dropped to 49.8%, and due to API issues it was unable to complete all 55 games. On Epoch AI's ECI index, Argon scored 165, on par with Fable 5.1, but ranked eighth on the

    Matched fieldsSummary、Body

    #ai-vertical#Evaluations / benchmarks#Google#DeepMind
  3. ARC PrizeSelected

    ARC Prize announces evaluation results of Gemini 4 Argon on the ARC-AGI series

    ARC Prize announces the validated results of Google Gemini 4 Argon on ARC-AGI: ARC-AGI-3 reaches 97.5% (provider adapter harness, cost $4.8K), and 49.8% under the standard harness (cost $5.8K); ARC-AGI-2 is 91.3%, at $0.32 per task; ARC-AGI-1 is 98.5%, at $0.16 per task. ARC Prize states that Google DeepMind is the second frontier AI lab to exceed 95% on ARC-AGI-3, and Argon's 97.5% is second only to GPT-6 Astra's 99.9%. The standard harness test failed to complete all 55 games due to API issues, and the high reasoning tier completed only 19 games so no score was published; costs are calculated at the current discounted price of $2 per million input tokens and $10 per million output tokens.

    harness, cost $4.8K), and 49.8% under the standard harness (cost $5.8K); ARC-AGI-2 is 91.3%, at $0.32 per task; ARC-AGI-1 is 98.5%, at $0.16 per task. ARC Prize states that Google DeepMind is the second frontier AI lab to exceed 95% on ARC-AGI-3, and Argon's 97.5% is second only to GPT-6 Astra's 99.9%. The standard harness test failed to complete all 55 games due to API issues, and the high reasoning tier completed only 19 games so no score was published; costs are calculated at the current disc

    Matched fieldsSummary

    #company#Evaluations / benchmarks#Reasoning#Google
  4. Story3 reports · 3 facts

    Chollet says Argon performs similarly to Astra on ARC-AGI-3

    François Chollet commented that Gemini 4 Argon performs similarly to Astra on ARC-AGI-3, with Google returning to the frontier. On the same day, ARC Prize published the verified results for Gemini 4 Argon on the ARC-AGI series: ARC-AGI-3 scored 97.5% under the provider adapter harness at a cost of $4.8K; under the standard harness it scored 49.8% at a cost of $5.8K. ARC-AGI-2 scored 91.3% at $0.32 per task; ARC-AGI-1 scored 98.5% at $0.16 per task. ARC Prize said Google DeepMind is the second frontier AI lab to exceed 95% on ARC-AGI-3, and Argon's 97.5% is second only to GPT-6 Astra's 99.9%. The standard harness test could not complete all 55 games due to API issues; the high-reasoning setting completed only 19 games, so no score was published.

    er the provider adapter harness at a cost of $4.8K; under the standard harness it scored 49.8% at a cost of $5.8K. ARC-AGI-2 scored 91.3% at $0.32 per task; ARC-AGI-1 scored 98.5% at $0.16 per task. ARC Prize said Google DeepMind is the second frontier AI lab to exceed 95% on ARC-AGI-3, and Argon's 97.5% is second only to GPT-6 Astra's 99.9%. The standard harness test could not complete all 55 games due to API issues; the high-reasoning setting completed only 19 games, so no score was published.

    Matched fieldsSummary

Thursday

Friday

  1. Story5 reports · 3 facts

    Google releases the Gemini 4 Carbon model

    On October 28, 2026, Google DeepMind released Gemini 4 Carbon, calling it its strongest model, ranking first with 59 points on the Artificial Analysis Intelligence Index v5.0, and scoring 88.2% on SEC-bench Pro, 54.8% on ExploitGym, and 71.0% on CWE-bench. The model targets long-horizon engineering tasks and was first opened to some cybersecurity defenders, then expanded through the Fairwind Program, and afterward made available to developers, enterprises, and consumers. On October 29, Arena.ai said Gemini 4 Carbon ranked second in Agent Arena with a net improvement of 16.2% and pushed the Pareto frontier forward; the same day, Yuchen Jin cited Artificial Analysis data to raise reservations, noting that the evaluation used the Antigravity CLI. On October 30, the Artificial Analysis Intelligence Index showed it in first place with 59 points, 1 point higher than Claude Opus 5.5. On October 31, Perplexity CEO Aravind Srinivas shared an early evaluation, saying it set a new SoTA on the WANDR benchmark.

    On October 28, 2026, Google DeepMind released Gemini 4 Carbon, calling it its strongest model, ranking first with 59 points on the Artificial Analysis Intelligence Index v5.0, and scoring 88.2% on SEC-bench Pro, 54.8% on ExploitGym, and 71.0% on CWE-bench. The model targets long-horizon engineering tasks and was first opened to some cybersecurity defenders, then expanded through the Fairwind Program, and afterward made available to developers, enterprises, and consumers. On October 29, Arena.ai

    Matched fieldsSummary

  2. Latent Space

    Elorian Releases DecoyCount Benchmark: Frontier Models All Score Below 50% on Visual Counting Accuracy

    Elorian and Mercor jointly released the DecoyCount benchmark, which tests whether models truly count objects or rely on visual shortcuts. Frontier models all scored below 50% accuracy, with the highest being GPT-6 Astra at 43.17%. Elorian says its visual reasoning model Glimpse scored 50.48% in a single run on 1,660 questions, making it the only model to exceed 50%, but this result comes from internal testing and did not use an agent or tool framework, whereas the six models in the paper each took the average of three runs. Elorian CEO Andrew Dai said the benchmark examines both vision and reasoning, and can distinguish whether a model failed to see the objects or miscounted them.

    Elorian and Mercor jointly released the DecoyCount benchmark, which tests whether models truly count objects or rely on visual shortcuts. Frontier models all scored below 50% accuracy, with the highest being GPT-6 Astra at 43.17%. Elorian says its visual reasoning model Glimpse scored 50.48% in a single run on 1,660 questions, making it the only model to exceed 50%, but this result comes from internal testing and did not use an agent or tool framework, whereas the six models in the paper each to

    Matched fieldsBody

    #newsletter#Evaluations / benchmarks#Multimodal#Reasoning

Wednesday

  1. Story1 reports · 1 facts

    Google releases world model Genie 4

    On October 27, 2026, Google DeepMind released the world model Genie 4, focused on real-time interactive simulation and introducing native audio to the series for the first time. According to the official introduction, the model is built on capabilities from the Gemini family: Gemini Omni is responsible for making the simulation match the appearance and behavior of the real world, while other Gemini models provide the complex reasoning needed to maintain the simulation's coherence; users can control scenes in real time using natural language, and each world also instantly generates ambient sound, music, and interactive dialogue. The next day, Google DeepMind further explained that Gemini 3.8 Flash breaks down goals in a loop and checks frames to maintain causal coherence across multi-step events, and that Gemini 4 Argon has also been used to jointly simulate tasks with Gemini Robotics; the company said it remains an early experimental model, that object permanence and complex physical interactions still drift during long runs, that all generated content carries a SynthID watermark, and that it is currently being previewed to trusted testers first. On October 28, Ethan Mollick posted that Genie 4 can run DOOM, and tested it generating a game-like environment as it goes, saying the experience is not perfect. On October 29, Peter Yang used Genie 4 to build 5 worlds in real time, arguing that world models are the next big direction for AI after LLMs.

    On October 27, 2026, Google DeepMind released the world model Genie 4, focused on real-time interactive simulation and introducing native audio to the series for the first time. According to the official introduction, the model is built on capabilities from the Gemini family: Gemini Omni is responsible for making the simulation match the appearance and behavior of the real world, while other Gemini models provide the complex reasoning needed to maintain the simulation's coherence; users can cont

    Matched fieldsSummary

Sunday

Wednesday

Thursday

Monday

Saturday

Thursday

Saturday