Blogger plans to run Ming-3.0-tts locally as a reading and dubbing assistant
The blogger plans to run the 0.5B Ming-3.0-tts voice synthesis and cloning model locally, to make a book-reading assistant for his son and German dubbing for videos. The reading assistant supports reading through the whole text and giving prompts at pauses or in specified areas; the dubbing solution combines ASR and multimodal video understanding to generate German dubbing and synchronize it with the visuals. He plans to put it into daily use in a week and open-source it.
View original postHave you guys seen Tan Tan Traffic? This show is hilarious, but honestly, a lot of the dialogue in it is hard to understand, like the one where a Fujian driver is looking for a hospital. I've always wanted to know what this Fujian guy is actually saying, and whether his friend's translation is correct. Now it's sorted: the Ming-3.0 speech series models are here and solved my problem. You can try them too! https://t.co/8Uy7ByTFmk
View original
看过谭谭交通么兄弟们，这个节目很搞笑，但是说话实说里边很多对话不是很能听懂，比如福建司机找医院这个。我一直想知道，这个福建人，到底说的是什么，他朋友的翻译到底对不对。现在好了，Ming-3.0语音系列模型来了，解决了我的问题，你们也可以去试试！ https://t.co/8Uy7ByTFmk
Body · AI translation
I want to try Ming-3.0-tts, this 0.5B voice synthesis and cloning model, because it's small enough that I want to run it locally!
Recently I've been doing some AI assistance for my son's learning. The typing game I open-sourced earlier counts as the first step (see my pinned post): https://t.co/B18Nghv5fI
Next I want to try some book reading and video dubbing. He has now moved from picture books to reading text books; his listening comprehension and pronunciation are both very good, but there are some words he doesn't know. I want to make him a book reading assistant that can read through the whole text, and can also give prompts at pauses or in specified areas.
As for video dubbing, most of the videos we usually watch are English videos, and his resources for learning German are relatively limited. If some of the videos he wants to watch had German, that would be great. Now that there are TTS models that can run locally, combined with ASR, multimodal model video understanding, and other capabilities, we can generate German dubbing and then match and sync it with the visuals.
No more talking, let's do it. I'm setting a flag: within a week it goes into daily study use, and then, as usual, I'll open-source it!