Anthropic 与 Vapi 网络研讨会问答：语音智能体如何选模型与写提示词

Vapicon 2026 is coming

Last week, Vapi co-founder and CTO Nikhil Gupta joined Marius Buleandra from Anthropic's Applied AI team for a discussion on Voice AI and Intelligence, a conversation on what can happen when a voice platform meets frontier models. The engaged audience sent in more questions than could be answered live. Below are answers and a recap of some of the most common ones. The audience was particularly interested in the makeup of the live ride-booking demo, which is embedded in this blog. We included specific questions around how it was configured and how it responded with such low latency.

1. How was the ride-booking demo agent configured? What models did it use?

The agent that booked rides live on the webinar is a real tool-calling agent running on Vapi, spun up in minutes. The configuration is simpler than most people guessed.

The model is Anthropic's Claude Haiku, the fastest of the models. The voice is Layla, a Vapi v2 voice cloned from an actual Vapi employee. Two platform settings were critical to making this agent successful. Smart endpointing waits through "uh" and mid-sentence hesitation instead of responding the moment a caller pauses. A conservative interruption threshold means the agent yields when the caller actually says something, not when the dog barks. The agent had five tools to help it take action: estimate a ride, book it, check status, cancel, and end the call.

Why Haiku instead of a bigger model? On a phone call, every millisecond of model thinking is silence the caller sits through. Haiku's time to first token is what made sub-second turns possible, and the task didn't require a more intelligent model. Ride booking is a bounded domain with well-defined tools, and the hardest reasoning in the demo, working backward from a flight time to a pickup recommendation, is comfortably within Haiku's range.

2. What makes a great voice agent prompt?

On the webinar, Nikhil mentioned that Vapi tested the same appointment booking objective, using one good prompt against one great prompt. The great prompt spoke 2.5x less. It knew the right thing to say, skipped confirmations it didn't need, and resolved the task faster. Tighter prompts are also cheaper and a better experience.

With the rise of the models, it is no longer necessary to provide a script, and more important to provide a goal: exact scripts perform worse. They don't give the model flexibility to stay conversational and go with the flow. What works is setting the overall objective, listing what information to gather, giving rough ordering guidance, and drawing the guardrails.

Marius added the hygiene side: smarter models need less prompting, and his practical tip is to paste your prompt into Claude and ask it to surface conflicts. Prompts grow every time someone patches a failure; nobody trims them. Best software engineering practices apply to prompting too.

3. Which model should power your voice agent?

When evaluating models, every model choice trades off latency, accuracy, and cost. The framework from the webinar for Anthropic models was as follows:

Claude Haiku, the speed runner. High volume, low latency: tier-1 support, quick answers, surveys, reminders. The most cost-effective model that still follows your guardrails; Claude Sonnet, the daily workhorse. The default for most production voice agents. Follows guardrails, stays on script, executes multi-step work, and is still fast enough to feel human. Patient intake, payment recovery, lead qualification; Claude Opus, the deep thinker. Highly intelligent but higher latency. Excellent for post-call work: CRM updates, sentiment, analytics, LLM-as-judge evals

For latency tuning, Marius's guidance: remember the biggest latency driver is output tokens, so a tighter prompt is also a faster agent. Use caching. And each new model generation moves the ladder: in his words, the new Sonnet is roughly as smart as the previous Opus, so you get big-model intelligence at small-model latency with every release.

4. Do voice agents handle multiple languages and accents?

In voice AI, language detection, mid-call switching, multilingual deployments, and non-English calls that feel natural have become normal. Voices can be specialized with different accents, dialects, and tones across many languages. Teams deploy this in two ways: a single agent that detects the caller's language and switches on the fly, or a squad that routes each caller to a dedicated assistant with its own prompt and voice. Accent coverage is built into each language model, so Brazilian and European Portuguese run on the same configuration with nothing extra to set up.

Agents running Claude on Vapi handled nearly 5M calls in the past 30 days across thousands of organizations, in 50+ languages.

5. How do you know a voice agent is ready for production?

This requires testing at scale, probing edge cases before launch, and benchmarking against human agents. Marius built his previous startup around this question. His approach: attack your own agent the way a bad actor would, fix the holes you find, then test again. Read the agent's script and enumerate everything it can and can't do, extract scenarios from that, and turn them into an evaluation set. Enough coverage gives you real confidence, and tests can be more aggressive than actual customers ever are.

Nikhil added the metrics side. Define failure modes and track them: the rate at which a caller has to ask for something twice should be under ~3% of calls, alongside CSAT. When a metric slips, root-cause it in the system prompt. The most common failure he sees is self-inflicted: teams patch the prompt daily after reviewing calls until it becomes an amalgamation of fixes with conflicting instructions. Evaluations catch the regression before production does.

6. How do you make an agent conversational, not robotic?

This can come in many forms: handling interruptions, filling silence during tool calls, cutting confirmation loops, and keeping context through long calls.

Start with latency as a constraint, not an afterthought. The most common mistake is porting a chat architecture with tool calls that take 10 to 20 seconds for voice. In chat, nobody notices; on a call, that's half the conversation spent in silence.

For long-running work, design the wait like a human would use it. While the booking runs, the agent can offer something relevant: "While we wait, I noticed you ride between home and work every day; we have a pass that makes that 10% cheaper." Chat is single-threaded; voice doesn't have to be.

7. When and how should an AI agent hand off to a human?

Voice agents can now hand off a call with full context, so the customer never repeats themselves, and they can pick the right moment to do it.

The trigger side is prompt design: define escalation conditions the same way you define guardrails, as explicit boundaries the model respects.

Once a trigger fires, there are multiple ways to pass context. One is a warm transfer: the agent briefs the human recipient, with an AI-generated summary, key context, or the full transcript. Or you push a summary into your system of record while the caller holds, and then complete the handoff within a single tool call.

8. How does building on a voice platform compare to other approaches?

The telephony stack underneath a voice call is decades old. Someone has to manage audio encoding, transcription latency, fallbacks when a provider degrades, and scaling compute when the model needs to reason. None of that is your product. Engineers face a bad tradeoff: build the full voice infrastructure yourself, which takes months of time and engineering resources, or buy a managed service that's expensive and takes control away from what your agent can do or say.

The platform approach takes you from zero to 90: telephony, conversational fluidity, call center integration, and human transfer are baked in. Your job is the last 10 points, the agent design and domain knowledge that make it yours. The proof point from the session: Amazon Ring used Vapi with Claude and went live in two weeks at 100% of call volume, with CSAT near human parity and still improving. With a platform, you can spin up a voice agent demo in minutes and a full production-grade deployment in just a few weeks.

Watch the full webinar replay.

Table of Contents

How an in-house review bot cut our PR merge time in half

Fast decisions alongside voice AI: early experiments with Jev

Build vs. buy: what you're really deciding with voice agents

What makes a great conversational AI experience?

Why model flexibility matters in voice

Built for the Ear: Designing Conversations for Voice

How we Bootstrapped the Voice Agents on the Vapi Homepage

AGI is here. Why am I still on hold?

Introducing Vapi Monitoring

Vibe code voice agents

Your Voice Agents Need Tests. Now They Have Them.

GPT-5.1 Just Fixed the Thing That's Been Bugging Me for Years

Introducing Squads: Teams of Assistants

GPT-5 Now Live in Vapi

How we solved latency at Vapi

What Is Signal Processing? Voice AI Definition Guide

How to Use Grok 3 in a Voice Agent

Unpacking LLM Temperature

How to Build a GPT-4.1 Voice Agent

What Is Gemma 3? Google's Open-Weight AI Model

Introducing Vapi Workflows

What Are IoT Devices? A Developer's Guide to Connected Hardware

Building a GPT-4.1 Mini Phone Agent with Vapi

What Is GPT? Understanding A Core Technology for Voice AI

What is a Phoneme? An In-Depth Look for Technologists

Launching the Vapi for Creators Program

Introducing Vapi Voices

Vapi x Cartesia: Ultra-Realistic Voice AI with Sonic 2.0

Start building