Search “Speech”

Sierra launches 𝜏-i18n multilingual voice agent evaluation benchmark

Sierra releases 𝜏-i18n, extending 𝜏-Voice to five languages: Spanish, Brazilian Portuguese, Hindi, Korean, and Mandarin, retaining the same airline, retail, and telecom tasks, policies, tools, and success criteria to enable cross-language comparison. Across 2,700 main calls, task completion rates for Spanish, Hindi, and Portuguese lag English by an average of 2.4, 3.8, and 6.4 percentage points, while Korean and Mandarin lag by 22.6 and 9.1 percentage points respectively; Korean is the weakest language among the three agents. In addition to task completion, 𝜏-i18n separately reports interaction metrics (response timing, selectivity, tool-call efficiency) and generation quality (naturalness, speech fidelity), and has open-sourced the codebase and the language pack framework LanguageFactory.

English translation is being prepared.

English translation is being prepared.

Google Releases EmbeddingGemma 2 Paper: A 744M-Parameter On-Device Multimodal Embedding Model

Google has released the EmbeddingGemma 2 paper, an Apache 2.0-licensed on-device multimodal embedding model that maps text, code, images, audio, and video into the same vector space. The model is built on Gemma 4 and tuned through contrastive training by learning from the larger Gemini Embedding 2; with 744M parameters, it surpasses models 2 to 12 times larger on multiple tasks. On the Pixel 11 Pro, the quantized text weights occupy about 191MB of active memory, while the full multimodal model takes about 567MB; the context window is expanded 4 times to 8K tokens, accommodating about 5.5 minutes of audio, 29 images, or 58 video frames; directly embedding speech scores 69.54 on the MSEB spoken query retrieval benchmark, higher than the 57.59 of transcribing first and then retrieving, and code retrieval on MTEB Code improves from 68.76 in EmbeddingGemma 1 to 78.68.

English translation is being prepared.

New study says AI model token consumption varies by up to 17 times depending on language

Research published last month by the AI Language Equity Research Alliance found that AI models consume vastly different numbers of tokens when processing different languages, with a gap of up to 17 times in one of the models. The study says token consumption already varies by 1.5 to 5 times among the 20 most commonly used languages on the internet, and the gap widens further when expanded to 200 languages; the researchers call this extra cost a hidden "tokenizer tax." Higher token consumption not only drives up costs but also slows response times and compresses the available context. The researchers recommend that AI labs publish evaluation results for each language and improve tokenizer design in the next generation.

Google DeepMind beats Minecraft with four Gemini 4 Argon agents

Google DeepMind published the paper "A team of Gemini 4 Argon agents beats Minecraft": four Gemini 4 Argon coding agents autonomously divided labor in a randomly generated Minecraft survival world, accumulating 605 deaths and writing 2,783 programs, defeating the Ender Dragon in 27 hours and 14 minutes. The experiment used Minecraft Java Edition 1.20.4, Easy difficulty, a random map, and no cheats; the agents read the environment via Mineflayer and wrote JavaScript to command characters, with roughly 24,000 model calls and 1.35 billion input tokens over the whole run, 91% of which came from context caching. The authors note that about 22% of game programs contained errors and the server restarted 6 times; the paper did not demonstrate stable completion through repeated tests on multiple random maps, and at discounted API prices the completion cost is estimated at about $520.