On May 29th, the independent variable robot team officially released the world's first embodied intelligent world model based on "event level prediction" - WALL-WM. This model breaks through the limitations of traditional embodied models in mechanically learning actions based on time frames, completely switching the prediction unit of the world model to semantic events, marking a new stage in the ability of robots to understand and execute tasks.
Industry pain point: Frame by frame learning leads to robots' rote memorization
In the current embodied intelligence industry, mainstream visual language action (VLA) models generally adopt a fixed pattern: given the current picture and instructions, predict fixed length action blocks.
This frame by frame fill in the blank training method has obvious flaws:
- Robots can only remember small physical movements and ignore the ultimate goal of actions
- Once faced with scene adjustments such as changing cups or tables, robots are prone to "collapse" due to a lack of generalization ability
The independent variable team pointed out in relevant academic papers that text, vision, and action information naturally have different time scales and manifold geometries in the real world.. Forcefully aligning within a single shared space can easily compromise the pre trained geometric priors.
Core breakthrough: a new mechanism centered on "events"
In response to this industry pain point, the WALL-WM world model has innovatively introduced an "event centered" training and execution mechanism.
Specifically:
- It divides the complex tasks of robots into events with clear semantics based on action boundaries, such as reaching out, grasping, shifting, etc.
- In actual operation, the model no longer rigidly calculates the next frame of the picture, but first pre rehearses "what kind of changes the next event will cause in the world"
- Then accurately translates this visual change into the motion trajectory of the robotic arm
Simple understanding: Traditional models are like copying a movie frame by frame, while WALL-WM is like understanding the plot paragraphs of a movie - knowing when to reach out.When to grab and when to shift.
Engineering reconstruction: Let the new architecture land stably in the physical world
In order to make this new architecture run stably in the real physical world, the independent variable robot team has carried out a series of hard core engineering reconstruction:
- Dual mode flexible switching: the system supports flexible switching between the "event mode" of variable length action output and the "unified mode" of real-time closed-loop control under the same base weight
- Unidirectional coupling division of labor growth: realize the one-way coupling division of labor between the video model and the action model, effectively avoid the precious dynamic prior in Internet video being biased by the action data prematurely
- 3D geometric awareness enhancement: for multi camera devices, the cone mask and tubular mask mechanism are introduced to force AI to establish cross view real 3D geometric correspondence capability
- Delay optimization: Through the new "ladder thinking chain decoding" technology, decoding delay is significantly reduced while retaining logical interpretability. Data pyramid: a rigorous data engineering system. At the data level, the independent variable team has built an extremely rigorous "data pyramid" system engineering. Bottom layer: relying on millions of level network universal videos to supplement visual priors. Top layer: focusing on real machine takeover and error correction data. Together with the following technical support: Four level hierarchical annotation./li>
- Dual clustering sampling
- Distributed "Muon" training system
- FP8 quantization on the deployment end
Achievement: WALL-WM not only leads in multiple indicators such as embodied video generation quality and 3D spatial perception, but also outperforms the real Core15L1 Achieved excellent task completion scores in the generalization scenario of benchmark testing.
Open Source Information
Currently, the open source code and homepage of the project have been officially made public.