Understand this story
On November 12, 2026, LiveKit published a real-world test on its product and engineering blog in which it deliberately made a hotel booking tool fail within a voice agent, comparing the performance of Gemini 4 Hydrogen and Gemini 3.8 Flash. LiveKit said that among 54 Hydrogen transcripts, no false reports appeared, while 3.8 Flash produced 2. In terms of latency, at the high setting Hydrogen's median first-sentence latency was 4.0 seconds, while 3.8 Flash's default setting was 5.2 seconds. LiveKit recommends using the medium setting for voice scenarios, and noted that at the xhigh setting the median first sentence requires 7.8 seconds, making it unsuitable for real-time calls.
AI-generated from public reports
Latest development · Report
LiveKit Tests Gemini 4 Hydrogen vs. 3.8 Flash Voice Agents: Tool Failures and Latency ComparedDevelopments
1 confirmed developments
LiveKit evaluates Gemini 4 Hydrogen's voice Agent tool failure behavior
Source:English text is being prepared.
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 12
LiveKit Product and Engineering BlogFirst-partySelected
LiveKit Tests Gemini 4 Hydrogen vs. 3.8 Flash Voice Agents: Tool Failures and Latency Compared
LiveKit deliberately made a hotel booking tool fail in a voice agent, comparing Gemini 4 Hydrogen and Gemini 3.8 Flash. Across 54 Hydrogen transcripts, no false reports appeared, while 3.8 Flash produced 2; at the high level, Hydrogen's median first-sentence latency was 4.0 seconds, versus 5.2 seconds for 3.8 Flash at its default level. The author recommends the medium level for voice scenarios; the xhigh level requires a median 7.8 seconds for the first sentence, making it unsuitable for real-time calls.