Jack Dorsey launches native speech model Dessa, which can run full-duplex conversations locally on an M4 Max MacBook
English text is being prepared.
Twitter co-founder Jack Dorsey has launched a native speech model called Dessa, which he says does not rely on text as an intermediary, is trained end-to-end on human conversation audio, can run locally offline at faster-than-realtime speed on a MacBook equipped with an M4 Max, and supports full-duplex conversation with the ability to interrupt and interject at any time. The original article says its training consumes hundreds of times fewer tokens than large language models, and that the entire training used only a single NVIDIA GB200 NVL72. The author also notes that this is currently only a small-scale research preview from a lab inside Block, that the demos were selected by the company, and that questions remain about how a pure-audio model performs on complex multi-step logical reasoning.
Why read this
AI score 69
The original article compares the traditional three-stage speech approach of ASR plus a text LLM plus TTS, and explains how end-to-end audio training differs for local full-duplex conversation.
Topics
View original postWe believe voice is the most natural way to interact with intelligence, so we created Dessa, the most authentic full-duplex speech model ever built. Talking with it feels like talking with a person…full of emotion, nuance, pauses, and a natural cadence. https://t.co/r0qFLdvd6c
View original
We believe voice is the most natural way to interact with intelligence, so we created Dessa, the most authentic full-duplex speech model ever built. Talking with it feels like talking with a person…full of emotion, nuance, pauses, and a natural cadence. https://t.co/r0qFLdvd6c
Body
卧槽，卧槽，这个真的值得一个卧槽兄弟们，Twitter 联合创始人 Jack Dorsey 掏出了秘密研发的原生语音模型，可以直接在M4 Max MacBook上本地运行，轻松跑通像真人一样的全双工对话。
Damn，我感觉现在的 AI 架构被彻底摧毁了啊， 不靠文字、没有文本大模型，端到端纯靠人类音频训练，像婴儿一样学会了听和说，
而且没有文字中介、无需调用云端 API，直接在 M4 Max 本地跑通像真人一样的全双工对话，
我觉得这可能正在宣告那个先转文字再合成语音的粗暴时代正式翻篇了。
很多人以为这又是硅谷做出来的又一个更加拟真、声音好听的 AI 语音玩具， 但如果你扒开它的底层架构仔细深入看一眼，就会发现它干了一件极其反骨的事：
它直接把今天大模型奉为圭臬的文字底座，一脚踹开了！！！
现在的语音助手，不管 GPT-4o 还是市面上各种实时语音，本质上全是三段式的拼装车，
把你的话转成文字（ASR），丢给文字大模型推理，再把文字合成语音吐给你（TTS）。
这种做法把人类语言里最宝贵的东西全阉割了， 你话里的冷笑、犹豫、叹气、哽咽和微妙的讽刺，在被翻译成冷冰冰的文本字符那一瞬间，就已经死透了。
Dessa 走了一条完全相反的硬核邪路， 零文字、零文本预训练大模型，端到端纯靠人类真实的对话音频训练。
它根本不知道世界上有文字这个东西，就像人类婴儿一样，完全凭借听和说的声音信号直接建立认知。
正因为彻底绕开了文本中介，它展现出了当前所有语音 AI 都不具备的原生全双工能力， 它可以一边听一边说，你能随时插话打断，它会在聊到好笑的地方半句话就笑出声，也能精准接住你语气里的叹息。
官方给出的数据更是打脸了当前的算力大力出奇迹流派， 它训练消耗的词量比大语言模型少了整整几百倍，整个训练只用了一台英伟达 GB200 NVL72，而且能在搭载 M4 Max 芯片的 Mac 笔记本上离线以超实时速度本地跑通。
以前我们总觉得大语言模型是通往一切智能的万能钥匙。
但这个实验狠狠撕开了一个盲区： 文字只是人类为了克服时间限制发明的低带宽压缩工具，声音和物理信号才是智能交互的高带宽本体。
当然咱们激动之余照例保持一份冷静哈哈，这目前还只是 Block 内部实验室的小规模研究预览，市面上的 demo 也全由官方精心挑选。
脱离了浩瀚的文本知识库，纯靠音频训练出来的模型在复杂多步长逻辑推理上到底能撑多久，还得打一个巨大的问号， 但我觉得至少它释放了一个极具穿透力的信号： 当大家都在用几万张显卡把文本上下文无限拉长时，最底层的智能交互革命，已经在向人类最原始的听觉本能大反扑了！