Google DeepMind's Genie 4 Adds Sound, Speech and Live Steering to Its AI-Generated Worlds

LONDON - Google DeepMind on Tuesday introduced Genie 4, a world model that generates interactive scenes in real time and, for the first time in the Genie line, produces its own sound. Ambient noise, music and spoken dialogue are generated along with the picture, and users can type instructions while a simulation is running to change what happens next, the lab said in a blog post.

Genie 4 is available now as a research preview to a group of trusted testers. Google says it will come to Google AI Ultra subscribers in Project Genie, its tool for generating interactive worlds from text or image prompts, and to early access partners of the Gemini API and Google AI Studio. The company did not give a date beyond “coming soon.”

Image: Google DeepMind

The model leans on Google’s Gemini family. DeepMind says Gemini Omni anchors the simulated worlds in real appearance and behavior, while other Gemini models handle the reasoning that keeps a scene coherent. Users move through a world with standard keyboard controls and steer it with text, asking for rain, a new camera angle or another person to walk in. Gemini adapts each prompt to fit the scene and, between inputs, keeps the world moving on its own.

DeepMind says the biggest jump is in people. Characters move more naturally, react to their surroundings and can hold a conversation that responds to what the user does. The model also simulates physics such as aerodynamics and collisions. Scenes stay consistent for up to two minutes, double the limit of Genie 3, TechCrunch reported.

Screenshot: @GoogleDeepMind on X

Underneath, DeepMind runs Gemini 3.8 Flash as an agent in the loop. As Genie 4 renders, Gemini watches the frames, breaks a goal into steps and checks that each one happened. In a robotics experiment, the lab paired Genie 4 with Gemini 4 Argon to simulate Apptronik’s Apollo humanoid playing checkers, with Argon feeding Genie the next step as the scene played out. The idea is to test how a robot or an AI agent would handle a situation before it happens in the real world.

“What we’ll increasingly see in the future are these hybrid world models, where you have a combination of Gemini writing code and text, and Genie ingesting that in real time and simulating reality at different levels of abstraction,” Tejas Kulkarni, a research director at Google, said during a press briefing, according to TechCrunch.

Image: Google DeepMind

Testers in film, journalism, education, gaming and engineering have been trying the model. DeepMind says filmmakers are using it as a draft mode for previsualization, running several takes of a scene and changing camera angles and dialogue on the fly. Photojournalists are testing whether a single archival photograph can become an explorable scene, and game developers are using it to prototype before they write engine code.

Image: Google DeepMind

The lab is candid about the limits. Details and objects can drift as a simulation runs longer or after several complex edits, and audio accuracy, legible text in the world and long-term consistency are still being worked on. Every video and audio clip Genie 4 produces carries SynthID, Google’s invisible watermark for AI-generated content.

Screenshot: @sundarpichai on X

World models have become a crowded and consolidating field. Runway released its second world model a couple of months ago, and AMD is acquiring Fei-Fei Li’s World Labs, TechCrunch noted.