Why is this Chinese company the one chosen by both OpenAI and Gemini for real-time voice interaction?

Before you've even finished speaking, it cuts in; when your emotions rise, it's still reading out answers by the book; when you want to interrupt and change the subject, it insists on finishing its last sentence...

What kind of conversation is this? It's pure one-upmanship.

To truly be good at chatting, it needs to understand emotions and subtext, keep up with the rhythm of your speech, and yield the floor to you promptly when you interrupt.

These little annoyances in experience may seem trivial, but behind them lies an entire chain: speech recognition, model understanding, real-time transmission, and terminal response.

If any link is slow, the feel of the conversation easily collapses.

Precisely for this reason, real-time interaction has become a direction jointly tackled by model, application, and hardware teams.

The just-concluded iRTE2026 Real-Time Intelligence Conference brought all these teams together:

Those researching next-generation interaction models, those building customer service Agents, and those making smart hardware were all discussing the same thing—

OpenAI's GPT-Realtime brings real-time reasoning and tool calling into voice models, enabling AI to further judge how to respond and which tools to call after understanding a request.

Google's Gemini Live lets conversation and deep thinking run in parallel, so users can keep communicating while complex tasks are processed in the background; GPT-Live, paired with Decisions, lets Agents trigger actions based on incremental transcription while the user is still speaking.

Leading vendors continue to advance real-time interaction, and iRTE has also become a window for observing how these technologies move into industry.

As early as 2020, this industry conference was upgraded from RTC to RTE, with topics expanding from real-time audio and video communication to richer real-time interactive experiences;

This year, an "i" (standing for Intelligent, real-time intelligence) was added to RTE, officially upgrading it to iRTE, further focusing on the integration of real-time interaction and generative AI.

Just one extra letter in the name, but far more work to take on.

From the special-track topics, the core direction of this year's iRTE conference is easy to see: more than ten forums and over 100 in-depth sharing sessions jointly probed the three real-world propositions that conversational AI must cross to land.

The first proposition is aligning model capabilities with business outcomes.

Meituan, Alibaba, MiniMax, and Agora discussed together on the same stage at the enterprise customer service Agent special track: no matter how excellent a large model's conversational performance is, if it cannot adapt to real business processes, it still cannot generate commercial value.

The industry urgently needs to bridge the gap between model approaches, engineering implementation, and business goals.

The second proposition: behind fluent conversation is a complete set of indispensable underlying technology foundations.

To achieve natural real-time conversation, one cannot rely on the large model alone. Technical experts from enterprises and universities/institutions such as StepFun, The Chinese University of Hong Kong (Shenzhen), Tsinghua University's Speech and Audio Technology Laboratory, Qwen Audio, Ant Ling, Xiaomi, and Soul held discussions at the conversational AI and audio AI technology special tracks around directions including interaction models, target speaker extraction, multi-speaker recognition, and real-time speech generation.

The industry has reached a consensus that any shortcoming in the above foundational technologies will directly damage the conversational experience.

The third proposition is the real-world test after technology moves from the lab to terminal hardware.

Rockchip, Uniview, and PETKIT Xiaopei proposed at the smart hardware and embodied intelligence special track that merely connecting large models to hardware is far from enough; on-device compute constraints and real-world environmental perception must ultimately be considered by returning to product value itself.

The main forum roundtable further pushed the discussion to the implementation level.

Chen Wei, founder of Xieyue Intelligence, believes that the key to embodied intelligence is to truly translate human intent into robot action. Robots must not only understand voice commands but also integrate multimodal information such as vision, voice, and their own state to complete perception, planning, and decision-making about the physical environment, and continuously adjust task execution based on environmental changes after actions.

Cai Guoxiang, co-founder of Rokid, proposed that the naturalness of conversation is the core determining the user experience of AI wearable devices;

Geng Xianglei, founder of Guanke Technology, pointed out that the latency of voice seats and the timing control of interrupt interactions will directly affect business results. Whether it is smart hardware or enterprise customer service scenarios, all tests ultimately converge into the interaction window of just a few seconds after the user speaks.

Models handle understanding and reasoning, while real-time networks, audio and video processing, and interaction orchestration also need to keep pace collaboratively.

Andrey Norkin, chief architect at NVIDIA and winner of the 2025 Technology & Engineering Emmy Award, shared AV2 next-generation video coding, filling in the key link of media delivery and exploring paths to guarantee high-quality video interaction experiences on top of higher compression efficiency.

From model capabilities to business closed loops, from terminal hardware to media transmission, the industry trends presented by iRTE are gradually becoming clear:

And the underlying foundation that supports this chain, determines the lower bound of interaction experience, and has the highest real-time requirements is precisely the field where Agora has deep expertise.

Looking along this chain, the work Agora must undertake is also changing.

In the past, video conferencing, online classrooms, and voice co-hosting mainly required smooth transmission of sound and images between people.

Now that Agents have joined in, speech recognition, model reasoning, tool calling, and terminal feedback all need to coordinate promptly during communication.

Agora founder and CEO Zhao Bin's "five-level conversation capability" framework proposed at the main forum further unfolds this change:

From L1's question-and-answer exchanges, to L2's grasp of interaction patterns and coordination of speaking and listening timing, to L3's understanding of individual intent and circumstances, to L4's understanding of multi-person relationships and communication boundaries, and ultimately toward L5's deep communication based on long-term context.

The most immediate test right now is handling pauses, interruptions, and task execution.

"Fast and slow brain" collaboration addresses exactly this kind of problem.

This framework keeps the interaction model in real-time conversation while the background model handles deep reasoning, search, and tool calling. Users can continue to add conditions and change requirements, and the system adjusts execution accordingly.

But after parallelization, the several models must still stay consistent: once a new requirement has been accepted earlier, the later stages cannot continue operating according to the old task.

Agora Conversational Model, demonstrated by Agora, adopts a hybrid conversation system with multi-model collaboration, organizing voice perception, full-duplex response, reasoning, tools, memory, and speech expression to coordinate continuous listening, intent judgment, and action.

The role of real-time infrastructure has also extended from connection and transmission to model collaboration during the conversation process.

The Conversational Agent System introduced by Zhao Bin combines the above capabilities with the global real-time network, audio processing, and inference infrastructure. Sound and images must be continuously transmitted, and turns, context, and business state must also be synchronized in a timely manner.

The direction of a distributed real-time inference cloud that he further proposed also includes running models close to users and dynamically selecting models, tools, and paths based on latency, load, and scenario.

These capabilities entering business and terminals are also supported by corresponding products.

The Conversational AI Engine connects models, the Agent runtime framework, and users, is compatible with cascaded and end-to-end architectures, and provides capabilities such as turn management, interruption handling, and attention locking;

The development kit helps hardware expand from voice conversation to visual and motion interaction;

Call Agent targets telephone business and provides tools such as line configuration, process orchestration, test deployment, and monitoring.

Developers can obtain access, AI programming assistance, and scenario example support through SDKs, Skills, CLI, and Recipes, and use open-source projects such as the TEN Framework to combine models, tools, and memory, reducing repetitive construction of foundational components.

From network and interaction processing to products and tools, Agora is organizing the capabilities required for people to communicate with Agents into a reusable infrastructure.

This also echoes the conference's shift from RTE to iRTE.

As real-time interaction begins to carry intelligence, the underlying system needs to enable models' understanding, reasoning, and action to continuously coordinate in real communication.

As for ordinary users, they may not care how many models are collaborating in the background or how many network nodes are involved. Our requirements have actually always been simple:

If the sentence is not finished, listen first; if the idea changes, keep up; if something is assigned, do it well —