ByteDance World Model Exposed: Zhang Yiming's Strategy and Its Divergence from Google Genie 4's Approach

In 2021, Zhang Yiming wrote in his resignation letter: "The impact of virtual reality, life sciences, and scientific computing on human life has already shown the dawn's first light; these require us to break through the inertia of our business to explore."

In June 2021, Caixin reported that Zhang Yiming had personally gone to Weifang to negotiate the acquisition of Pico, and that Zhang Yiming had shown acquisition intent upon meeting. The acquisition later became a landmark event in this round of metaverse investment fever, but Caixin specifically noted in its news report that at the time the two sides did not disclose an intent to "enter the metaverse."

Five years later, in June 2026, ByteDance's world model was exposed; the tip-off said the world model was one of ByteDance's four major AI priorities for 2026, with the goal of releasing at least one version by the end of the year, benchmarked against Genie 3.

The project is based on Seedance; users enter through Pico headsets, say a sentence or make a gesture, and the cloud generates a world that changes accordingly.

The "virtual reality" Zhang Yiming wrote of did not refer to the metaverse, but to something more ambitious: the world model. His resignation letter can in fact also be understood as a letter of foretelling.

In September 2026, another tip-off said that Zhang Yiming, who had by then become China's richest person, was still personally overseeing a real-time spatial video model at ByteDance, coordinating different business units and channeling AI resources and compute toward the project. This shows how much he cares about world models.

It is unclear how far ByteDance has gotten in studying Genie 3, but Google struck first, releasing the world model Genie 4 on October 28. This undoubtedly put some pressure on Zhang Yiming.

Compared with previous generations, Genie 4's biggest feature is that users can continuously "intervene" in the world. In the demo Google released, you can step into Genie 4's world, make it rain, turn the camera, and make a person appear on a street corner. The wind has sound, and characters respond.

People who took part in testing said that what you set for Genie 4 is no longer just a single shot; you can assign it an event that is about to happen, and Genie 4 will automatically generate everything corresponding to that event.

What is Genie 4?

Rather than calling Genie 4 a world model, it would be better to call it a game.

Previous world models were actually a kind of video model, pre-generating the images you might see in the future for you to watch; Genie 4, by contrast, generates the next frame based on your actions.

If you make the protagonist in the video turn left, what a traditional video model can do is regenerate a video of him turning left.

But with Genie 4, you press WASD and the character moves and turns following your operation; at that point, the model needs to determine how the camera, character, objects, and spatial relationships should change next, and then present the result to you in real time.

The former generates a piece of content; the latter simulates the consequences of actions.

Google has been walking this road for several generations.

Genie 1 in 2024 learned latent actions from internet videos without action annotations, allowing people to control the generated environment frame by frame. Its importance was not how refined the images were, but that the model began learning the relationship between "actions" and "changes" from video.

Genie 2 the same year turned a single image into a 3D space that a person or AI could operate with a keyboard and mouse, and began showing object interaction, character animation, gravity and lighting, as well as the ability to re-render scenes outside the field of view.

By Genie 3 in 2025, the world advanced to 720p at 24fps and could maintain content consistency for several minutes. Besides navigating, users could change the weather and add characters and objects through prompts. Video first became controllable, then enterable, and finally continuously intervenable.

Genie 4's main improvement is making this world more worth "playing."

You can use natural language to change the weather in real time, move the camera, and introduce a person or an object. Gemini will understand your instructions in combination with the current scene, rather than tearing down and rebuilding the world every time.

Between your two inputs, it will also continue to monitor and guide the scene, keeping the world moving forward.

Google emphasizes that Genie 4 includes physical simulations such as aerodynamics and collisions, as well as more natural character movements and reactions to the environment.

In addition, Genie 4 is also the first model in the Genie series with sound. It can generate wind, waves, and background music in sync with the visuals.

Previously, humans kept telling the world "what happens next." Now, Genie 4 is beginning to understand "how this event should happen step by step."

Gemini 3.8 Flash observes the frames generated by Genie 4 in the loop, breaks complex goals into steps, checks whether each step is completed, and adjusts the plan after the world changes or the user intervenes.

Of course, world models do not have only the path of "letting people go in and play games."

For example, Nvidia's Cosmos uses a world foundation model for simulation, reasoning, and data generation, for training robots, autonomous driving, and industrial vision.

Autonomous driving needs to be trained on rare traffic scenarios such as extreme weather and complex lighting.

Simulating such scenarios in reality is not only costly but also unsuitable to create deliberately; if a simulated world can generate these situations, machines can first train, test, and validate inside it.

Robots also need to understand the consequences of actions. For example, if it picks up a cup, will it slip? After the palm collides with the cup, how does the object move? If the cup is placed in a different position, can the original strategy still work?

This path requires not only realistic visuals, but also controllable synthetic data, physical interaction, action consequences, and repeated validation.

Google is likewise exploring the combination of Genie 4 and Gemini Robotics, but the hallmark of this generation is still entertainment, with an emphasis on immersion and interactivity.

Ultimately, Genie 4 is only a preview version. Google says it can maintain consistency only in short sessions. When runtime is too long, or after multiple complex modifications, visual details and object permanence may still drift. At the same time, the model's complex physics, real-time audio, spatial precision, and long-term consistency remain unsolved.

ByteDance laid plans early

ByteDance's world model has similarities with Google's.

In 2024, after Zhou Chang, the technical lead of the Tongyi large model, joined ByteDance from Alibaba, he was put in charge of research related to world models.

At the time, however, ByteDance judged internally that both the technical path and the commercialization scenarios for world models were not clear enough, and that the more important thing was to first win the battle of video models.

By 2025, ByteDance began forming a small VLA research group. The team of Li Hang, head of ByteDance AI Lab, mainly trained on simulation data, while the team of Wang Wenqian, a multimodal researcher at Seed, mainly trained on natural data.

After the 2026 Spring Festival, Seed set up another 3D simulated world model group, led by former Meta FAIR researcher Fan Haoqi, reporting to Zhou Chang; the original two VLA groups were merged and also reported uniformly to Zhou Chang.

This formed two paths: VLA aimed at embodied intelligence, pursuing real actions and environmental responses; 3D simulation aimed at entertainment and games.

ByteDance's training data budget for the world model reached tens of millions of yuan, covering modalities such as VLA, long video, and 3D. A data supplier said the investment could reach 3 to 4 times that of other vendors.

Yet the money was spent and the people were hired, but results just would not come. According to reports, in early 2026, ByteDance's internal evaluation showed that the overall performance of its world model was still about 10% behind the global SOTA at the time.

Just like Google, ByteDance has technical support for building world models.

Seedance 2.5 can generate video content up to 30 seconds long in a single pass, and can continue the story through multiple rounds of prompts, thereby outputting a complete short film.

It can generate video by referencing multiple kinds of materials, and can also modify by timestamp the character poses, motion trajectories, and camera movement; it can also first build a white model and then generate video based on the white model.

Video models in the past were essentially gacha: once the prompt was sent out, the model's output could be good or bad. Seedance 2.5 is a combination of AI video generation plus AI editing, and the output quality is also more stable.

A world model needs not only continuous frames; it also needs each object inside to be three-dimensional, so that you can view it from different angles.

Hence Seed3D. It can turn an image into a 3D model with geometry, textures, and PBR materials, and after being imported into Isaac Sim, it can also be used for physical simulation and robot training.

In other words, the assets generated by Seed3D are 3D assets that comply with physical rules and can be used for action testing.

Finally, there is SeedRealtime, a native full-duplex audio-video large model. It is a monolithic-architecture, end-to-end real-time interaction model that models audio + video + text + temporal streams.

No matter how beautiful the visuals, if a world cannot understand what people are saying or know when to respond, it is still just a video.

On October 11, Tim of Media Storm showed such a video: the user first generated a room from a miniature perspective with one sentence, then rode a paper airplane with keyboard and mouse, flying freely through the scene.

Want a spoon, and the world generates a spoon, and you can even climb along the spoon. Then draw an "anywhere door," and the originally unrelated room and outer space are connected together.

Although ByteDance did not officially disclose specific information about this world model, it gave the outside world its first look at the prototype of ByteDance's world model. It emphasizes spatial exploration, and also emphasizes modification and interaction at any time.

In the 2026 Hurun Rich List released on October 27, Zhang Yiming became China's richest person, with wealth of 670 billion yuan.

Zhang Yiming hopes this model can connect four things: AI models, cloud computing, content platforms, and Pico hardware. Ultimately, he hopes the world model will become ByteDance's new flywheel.

Seed is responsible for generating the world, Volcano Engine provides cloud computing power, Jianying and Jimeng lower the creative threshold, Douyin, livestreaming, and short dramas handle distribution of the experience, and Pico provides the spatial entry point.

Pico's advantage lies in being lightweight and cheap.

Traditional XR devices need to stuff a large amount of computing into the headset, which makes the headset heavy and expensive, such as Apple's Vision Pro. Zhang Yiming wants to put complex computational generation in the cloud, leaving the headset responsible only for perception, transmission, and display.

Costs do not disappear; they merely shift from the headset to the data center.

Therefore, the more users there are, the higher the cost of concurrent computing. The slightest network fluctuation will cause latency to directly destroy immersion.

At present, the generation metric of ByteDance's world model is about 20fps, which cannot be called smooth; at best it can only be considered "not stuttering."

From "recommend the next one" to "generate the next second"

Zhang Yiming is not the only one who can build world models. Fei-Fei Li, Yann LeCun, DeepMind, Nvidia, and a large number of robotics and autonomous driving teams are all building world models from different directions.

They are all technical experts, but for world models to move from the lab to the masses, more than technology is needed. It also requires low-friction interaction, a creator ecosystem, content distribution, and so on.

What Zhang Yiming is best at is not inventing every underlying technology; he is better at bringing technology to market.

Take the recommendation algorithm that ByteDance was founded on. This was obviously not invented by Zhang Yiming, but Zhang Yiming combined this technology with ByteDance's existing product organization, compressing the act of "finding the next piece of content" into a single swipe.

You move your thumb, and the next piece of content you might like arrives.

Users do not need to understand what a recommendation system is, nor do they need to learn a set of operating methods. Swipe once; if you don't like it, swipe again.

World models likewise need such a way of reaching users.

English text is being prepared.

Zhang Yiming shortened the distance between people and content, and the world generated by a world model is also just content.

Likewise, organizing creation, distribution, and feedback are also things Zhang Yiming is good at.

A bit of trivia: infinitely generating worlds does not equal infinitely generating good experiences.

Randomness does not equal richness, realism does not equal fun, and interactivity does not equal users being willing to interact. A space that can extend infinitely, if it lacks goals, characters, conflict, and rhythm, is ultimately nothing more than a space.

Therefore, the new "world" needs creators, and it needs a large number of creators. What they need to create are initial settings, character relationships, rules, and a scene that allows users to explore and interact.

Once generation is no longer scarce, "what is worth generating" will become scarce.

ByteDance's existing creators, tools, and distribution system may make it possible for a kind of "world model designer" to become a profession like AIGC creators. These people are responsible for making the world hold together, and then continuously revising it based on user behavior.

A deeper change is from "recommending the next one" to "generating the next second."

A recommendation system faces a content library that already exists.

You have just finished watching a video. Based on your preferences and user profile, the system predicts what you are most likely to watch next, and then picks the next one from the library.

A world model, by contrast, faces a next second that has not yet fully happened.

You take a step forward, turn left, say a sentence, or pick up an object. The model needs to predict how the environment should change after your action, and generate this moment in real time.

On Douyin, ByteDance distributes already-produced content based on behavior. In a world model, ByteDance can produce the content itself in real time based on each behavior.

Broadly speaking, this is still a kind of "recommendation," because both are predicting what should appear before your eyes. The only difference is that in the past, what was recommended to you was content someone else had already made. In the future, what is recommended to you will be a result that did not exist before.

This is also why the assets Zhang Yiming left behind during the metaverse era are still worth reexamining today.

The world awaits Zhang Yiming. But I just want to say: hurry up and bring the world model to the table, I can hardly wait.