Google DeepMind Releases World Model Genie 4, Supporting Native Audio Generation
Google DeepMind has officially released its new-generation world model Genie 4. Compared with Genie 3, it improves visual realism, physical simulation, and real-time interaction capabilities, and for the first time supports native audio generation, capable of synchronously generating ambient sound effects, music, and character dialogue. Users can control the camera or character movement with the keyboard, and can also use natural language instructions to adjust weather, switch viewpoints, and add characters or objects in real time, without needing to regenerate the entire scene. Genie 4 integrates capabilities from the Gemini series: Gemini Omni provides real-world visual and behavioral knowledge, while Gemini 3.8 Flash handles task planning and scene guidance. It is currently available as a research preview to some invited testers, with plans to later open it through Project Genie to Google AI Ultra subscribers and some developers.
Why read this
AI score 80
Genie 4 adds native audio generation on top of visual and physical simulation, and outlines an experimental path for training robots in collaboration with Gemini.
Topics
Body · AI translation
English text is being prepared.
On October 27, Google's Google DeepMind officially released its next-generation world model, Genie 4. Compared with the previous generation Genie 3, the new model further improves visual realism, physics simulation, and real-time interaction capabilities, and for the first time supports native audio generation, allowing it to generate ambient sound effects, music, and character interaction dialogue in real time while constructing virtual scenes.
According to Google, Genie 4 can continuously generate interactive dynamic environments. Users can control the camera or character movement via keyboard, or input natural language instructions to adjust weather, switch perspectives, and add characters or objects in real time without regenerating the entire scene. In terms of physics simulation, the model strengthens its simulation of aerodynamics, collisions, and character movements, enabling virtual characters to respond accordingly to environmental changes.
In addition, audio generation is also a key focus of this upgrade. Genie 4 can synchronously generate wind sounds, water sounds, music, and character dialogue based on scene content, and virtual characters can also adjust their language and behavior in real time according to user input and scene events, achieving coordinated generation of visuals, sound, and interaction.
In terms of technical architecture, Genie 4 further integrates the capabilities of the Gemini series models. Among them, Gemini Omni provides real-world visual and behavioral knowledge for virtual environments, while Gemini 3.8 Flash handles task planning and scene guidance, continuously observing generated frames, breaking down goals, and adjusting subsequent steps to improve the coherence of complex event simulation.
Google also demonstrated an experiment in which Genie 4 and Gemini 4 Argon collaboratively simulate robotic tasks. In the demo, Gemini is responsible for decomposing tasks, and Genie 4 generates the corresponding environment according to real-time instructions, simulating the Apptronik Apollo humanoid robot performing an international draughts task. Google believes that this approach of combining reasoning models with world models is expected to provide more flexible virtual training and testing environments for robots and AI agents.
In terms of applications, Genie 4 can be used in areas such as game prototyping, film and television previsualization, historical scene reconstruction, and industrial simulation. However, Google said that the model is still in an early research stage and still has limitations in long-term scene consistency, complex physical interactions, audio accuracy, and spatial precision.
Currently, Genie 4 has opened a research preview to some invited testers, and there are subsequent plans to open it to Google AI Ultra subscribers and some developers through Project Genie.
Submissions/Tips: tougao@youxituoluo.com
Manuscripts/Business Cooperation: Lin Nan (WeChat 19250561593)
Join the industry exchange group: Lin Nan (WeChat 19250561593)
Metaverse Digital Industry Service Platform
English text is being prepared.