Understand this story
On October 22, 2026, Sierra released the τ-Entity benchmark, designed to test whether voice agents can accurately capture, recognize, and correct errors in entity details such as names, phone numbers, addresses, and email addresses. The benchmark includes 200 tasks, covers ten categories of entities, and features a six-language leaderboard. According to the leaderboard, GPT Live 1 achieved a 60.3% accuracy rate in regular calls, but only 34.0% of tasks were passed under all three conditions: regular, strong noise, and strong speech. Gemini 3.8 Live and Grok Voice Think Fast 2.0 had all-condition pass rates of 19.5% and 24.0%, respectively. The paper also found that prescribing a "ask—spell/repeat—confirm correction—submit" workflow can increase the all-condition success rate by 14 to 31 percentage points, at the cost of 21 to 28 extra seconds per call. There are currently no follow-up developments.
AI-generated from public reports
Developments
1 confirmed developments
Sierra launches voice Agent entity detail benchmark 𝜏-Entity
Source:Sierra：Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 22
Sierra：BlogFirst-party
Sierra launches the 𝜏-Entity voice agent benchmark: GPT Live 1 passes only 34.0% of tasks under all conditions
Sierra releases the 𝜏-Entity benchmark, using 200 tasks and ten entity categories (names, phone numbers, addresses, emails, etc.) to test whether voice agents can accurately capture, recognize, and correct detail errors. On the six-language leaderboard, GPT Live 1 leads, with 60.3% accuracy on regular calls, but only 34.0% of tasks pass under all three conditions—regular, heavy noise, and strong voice; Gemini 3.8 Live and Grok Voice Think Fast 2.0 score 19.5% and 24.0%, respectively. The paper also finds that mandating a "ask—spell out/repeat—confirm correction—submit" workflow can raise the all-condition success rate by 14–31 percentage points, at the cost of 21–28 extra seconds per call.