AI News

Arena Evaluation: Gemini 4 Hydrogen Has the Fastest Step Latency Among the Agent Arena Top Ten

ArenaUpdated Nov 13 02:34

AI summary

Arena.ai states that Gemini 4 Hydrogen is the fastest model among the Agent Arena top ten, with a typical step latency of 7.3 seconds, about 2.7 times that of Claude Sonnet 5.5 and 3.1 times that of GPT-6.1 Sol. In Agent Arena sessions, 82% of Hydrogen's tasks were completed within 10 minutes, compared with 55% for GPT-6.1 Sol and 44% for Claude Sonnet 5.5; within 30 minutes, the completion rate was 97% for Hydrogen and 81% and 78% for the latter two. Arena says Hydrogen's overall improvement is comparable to these two models and that it did not trade quality for speed. Its measurement counts a single model API call within an Agent Mode task, timed from the first model request to the last streaming response, excluding tool execution.

Why read this

SelectedAI score 79

Arena compares the three models using two metrics, step latency and task completion time, showing how speed differences translate into actual waiting time.

Topics

Tags

Body

Gemini 4 Hydrogen by @GoogleDeepMind is the fastest model in the Agent Arena top 10.

A typical Hydrogen step takes 7.3 seconds, about 2.7x faster than Claude Sonnet 5.5 and 3.1x faster than GPT-6.1 Sol.

Faster steps mean less time waiting on tasks. Across Agent Arena sessions, Hydrogen spends under 10 minutes on 82% of its tasks, compared with 55% for GPT-6.1 Sol and 44% for Claude Sonnet 5.5. Within 30 minutes, that rises to 97%, versus 81% for Sol and 78% for Sonnet 5.5.

And it isn't a quality trade-off: Hydrogen's net improvement is on par with both models.

How we measure it: A step is one model API call in an Agent Mode task. All times run from the first model request to the last streamed reply, without tool execution.

Source

Arena