Google DeepMind Releases Genie 4 Real-Time World Model with Native Audio Generation
Google DeepMind has released the Genie 4 world model as a research preview to trusted testers; it can generate explorable worlds in real time from text and image prompts, and natively generate ambient sound, music, and character dialogue. The model introduces Gemini 3.8 Flash as a planner within the simulation loop, responsible for breaking down goals, inspecting frames, and replanning, which DeepMind calls an early step toward hybrid world models. Genie 4 is hosted only on Google's platform and will later be opened to AI Ultra users and Gemini API partners; all video and audio carry SynthID watermarks.
Why read this
AI score 68
Genie 4 puts the Gemini planner into the simulation loop, allowing readers to understand how world models support agent training environments.
Topics
Body
Add as a preferredsource on Google
Google DeepMind has released Genie 4 as a research preview for trusted testers. The Genie 4 world model generates explorable worlds from text and image prompts in real time. It also generates the soundscape: ambient noise, music and character dialogue. Users move with keyboard controls and steer events with plain-language prompts. The bigger change is architectural. A Gemini model now runs inside the simulation loop as a planner. That pattern is important for anyone building training environments for agents or robots.
TL;DR
Runs on: Google-hosted only (Project Genie web app, Gemini API early access).
Performance: Photorealistic worlds with native audio and live text steering.
Best: Scene consistency for up to 2 minutes, double Genie 3, per TechCrunch’s briefing.
Worst: Sessions last a few minutes, and objects drift after repeated prompt edits.
What is Genie 4?
Genie 4 is a world model from Google DeepMind that simulates interactive environments frame by frame. It is not a video clip generator. Each frame responds to user input as it arrives.
TEXTTABLELAYOUTIs your PDF parser actually getting it right?omni_parse_bench · scores text, tables & layout · open sourceStar on GitHub→Sponsored
DeepMind states that the model is natively multimodal. Audio, music and dialogue are generated together with the visuals. It is DeepMind’s first real-time world model to generate audio natively.
It succeeds Genie 3, announced in August 2025. Genie 3 ran at 720p and 24 frames per second, with visual memory reaching back 1 minute.
How does Genie 4 use Gemini?
Genie 4 borrows from 2 parts of the Gemini family. Gemini Omni anchors the look and behavior of each world. Other Gemini models handle the reasoning that keeps a simulation coherent.
The runtime loop works like this:
Interpret: When you type a prompt, Gemini adapts it to fit the running scene.
Plan: Gemini 3.8 Flash breaks a goal into a sequence of steps.
Verify: It reviews incoming frames to confirm each step succeeded.
Adapt: If the world shifts or you intervene, it revises the plan.
Between user inputs, Gemini also steers the scene so the world keeps progressing. DeepMind team frames this as an early step toward hybrid world models. In that design, a language model handles structure and narrative, while Genie renders pixels and sound.
What can you do inside a Genie 4 world?
Steer with text: You can change the weather, move the camera or add a new person or object mid-session.
Simulate physics and people: DeepMind cites aerodynamics and collisions. It calls human motion the biggest leap, with characters that react to their surroundings.
Hear the world: Every world carries ambient audio. Characters can also hold conversations that react to your inputs in real time.
Test robot tasks: In one research demo, Gemini 4 Argon split a checkers task into steps. Genie 4 received each step as a text caption and simulated Apptronik’s Apollo humanoid playing the game.
How does Genie 4 compare with other world models?
DeepMind released no benchmark tables for Genie 4. The comparison below uses only disclosed specs from each project’s own source.
FeatureGenie 4Genie 3Runway GWM Worlds 2Skywork Matrix-Game 3.0
ReleasedOct 27, 2026Aug 5, 2025Sep 3, 2026Mar 27, 2026
ParametersNot disclosedNot disclosedNot disclosed5B (plus 2x14B MoE variant)
Resolution / FPSNot disclosed720p / 24 fps720p / 24 fps720p / up to 40 fps
Native audioYes, incl. dialogueNot reportedYes, 48 kHz, incl. speechNot reported
Mid-session text controlYes, Gemini-mediatedYes, promptable world eventsYes, timestamped text actionsKeyboard and mouse actions
LLM planner in loopGemini 3.8 FlashNoNo (optional external harness)No
ConsistencyA few minutes; 2 min per TechCrunchA few minutes; 1 min visual memoryNo preset length; memory imperfectMinute-long memory
HardwareGoogle-hosted, not disclosedGoogle-hosted, not disclosedRunway-hosted, not disclosed8 GPUs for DiT plus 1 for VAE
Access / licenseTrusted testers; AI Ultra and API soonProject Genie (AI Ultra)Research preview, contact formMIT, open weights
Who can use Genie 4, and when?
All Genie 4 video and audio will carry SynthID watermarks. DeepMind states that it ran internal safety evaluations before the preview.
Testers so far span creative industries, journalism, education, gaming and engineering.
Reported uses include:
Filmmakers running multiple live ‘takes’ for pre-visualization.
Photojournalists turning a single archival photo into a grounded simulation.
Game developers testing styles and mechanics before writing engine code.
Enterprise partners generating zero-to-one simulation scenarios.
What are Genie 4’s limitations?
DeepMind lists several open problems on the Genie 4 model page:
Visual details and object permanence drift in longer sessions, after complex prompt edits, or during navigation.
Interaction is limited to a few minutes, not extended sessions.
Real-world locations are not simulated with perfect accuracy.
Legible text usually appears only when it is in the input description.
Real-time audio accuracy and multi-agent interactions remain unsolved.
Key Takeaways
Genie 4 generates world, ambient audio and dialogue together, in real time.
Gemini 3.8 Flash plans, checks frames and replans inside the loop.
No parameters, resolution, frame rate or benchmarks were disclosed.
Access starts with trusted testers, then AI Ultra and Gemini API partners.
Drift and short sessions remain the main practical limits.
Michal Sutter+ postsBio ⮌
Michal Sutter is a data science professional with a Master of Science in Data Science from the University of Padova. With a solid foundation in statistical analysis, machine learning, and data engineering, Michal excels at transforming complex datasets into actionable insights.
Michal SutterMirendil Launches Fern 0.1, Its First Model, With 10M-Token Context
$uv pip install omni-parse-bench✓text✓tables✓layout→ scored per pageStar on GitHub→Sponsored
RELATED ARTICLESMORE FROM AUTHOR
Prime Intellect Releases Prime Mega MoE: Trainable Fused MoE Kernel, 3.11x Faster Forward Pass
NVIDIA Releases Nemotron 3.5 Transcribe: An Open 30B MoE Model for 40-Language Transcription with ~4B Active Parameters