←News detail

Shanghai ObserverUpdated Oct 26 09:25

DEEP Robotics releases five embodied brain achievements including DR-WM-1.0 and DR-VLA-2.0

AI summary

DEEP Robotics intensively released five embodied brain achievements: the world model DR-WM-1.0, the embodied foundation model DR-VLA-2.0, the embodied agent DR-Agent, the policy optimization framework DR-APO, and the data closed-loop platform DR-Benchmark. In post-training for embodied visual navigation, DR-APO raised the NavDP simulation success rate from 70.33% to 89.55%, and raised the Lynx M20's real-world navigation success rate from 25% to 65%. The core dataset of DR-Benchmark, LynxScenes, is based on a cumulative 833.6 hours and 3,294.7 kilometers of real-world operation data collected by the Lynx M20.

Body · AI translation

The development of embodied intelligence has entered a new stage of large-scale deployment in real-world scenarios. Recently, DEEP Robotics released five research results in quick succession centered on its self-developed embodied brain DEEPRobotics-Brain: the world model DR-WM-1.0, the embodied foundation model DR-VLA-2.0, the embodied agent DR-Agent, the policy optimization framework DR-APO, and the data closed-loop platform DR-Benchmark. The five results cover the full technical chain of the embodied brain and outline the clear architecture of DEEP Robotics' embodied brain. World model DR-WM-1.0: Introducing physical constraints to foresee a realistic future. At present, generative world models used for robot decision-making face a gap: the aesthetic imagination capability in pixel space is difficult to convert into the physical imagination capability needed for embodied intelligence, and DR-WM-1.0 released by DEEP Robotics directly addresses this gap. DR-WM-1.0, released by DEEP Robotics in collaboration with Zhejiang University, Beijing Institute of Technology, Tsinghua University, MIT, and the University of Illinois, is the first world model in the quadruped robot field to introduce physical properties such as friction, roughness, and stiffness, enabling the world model's predictions of future frames to be not only realistic but also consistent with physical laws. The overall pipeline of DR-WM and locomotion performance under different contact conditions. This provides an analyzable and verifiable technical path for narrowing the gap between visual realism and physical realism. Take substation inspection after rain or snow as an example: faced with a slippery shortcut and a detour route, the robot can first "rehearse" each separately, predict the possibility of slipping, posture instability, and blocked progress, and then make a decision by combining safety risks with passage efficiency. Embodied foundation model DR-VLA-2.0: Understanding human social rules so robots truly "understand". With rehearsal capability in place, robots can move toward real roads and workstations—but the next test follows immediately: how can robots better integrate into human life and understand human social behavior? Traditional robot teaching relies on teleoperation, which has high collection costs and is difficult to cover rich environments and interaction contexts. DEEP Robotics, together with researchers from MIT, the University of Illinois Urbana-Champaign, Tsinghua University, and Zhejiang University, explored a quadruped robot navigation learning path of "action-label-free human video pretraining—robot teaching adaptation." The EgoLynx dataset built by the team integrates 10,000 hours of first-person human web videos and 2,000 robot teaching clips, and innovatively uses the DISR module, enabling the robot to learn behavior patterns from massive videos, understand traffic rules and navigation rules, understand human intentions, and learn social etiquette, further improving the anthropomorphism of embodied intelligence. Overview of the DR-VLA method. On the left are first-person human videos, in the middle are three experts and robot data, and on the right are future scene representations and predicted trajectories. Human videos provide understanding and prediction supervision, while robot demonstrations further provide action supervision. Technically, this is the first quadruped robot navigation paradigm based on domain-invariant scene representation (DISR). In the EgoLynx-Robot test set evaluation, the comprehensive navigation score QNS improved by about 5%, and the final heading error decreased by 14.6%. With the help of this research, robots in scenarios such as "last-mile" delivery in communities and parks are expected to combine route instructions to predict changes in pedestrians and obstacles and integrate more naturally into everyday spaces where people and vehicles mix. Beyond walking, fine manipulation is the real challenge for robot deployment. DEEP Robotics proposed its self-developed atomic motion coordinate large model, which for the first time incorporates language control and force feedback into the same motion coordinate framework, decomposing robotic arm motion into 13 atomic coordinates and establishing a "unified coordinate language" for motion, enabling robots to "understand, see, feel, and adjust." The related capabilities can be extended to power scenario operations such as power cabinet operation, connector insertion, and part alignment, and have important practical deployment value. When the image and state are fixed and only the positive and negative atomic prompts are changed, the end-effector translation and rotation trajectories produce responses in opposite directions. Policy optimization framework DR-APO: Success rate increased to 89.55%. For model capabilities to continue getting stronger, one question cannot be avoided: how to train? DEEP Robotics, together with Zhejiang University, proposed DR-APO—the first representation-decoupled general generative policy reinforcement learning framework. This framework overcomes the difficulty in traditional native fitting where local improvement signals are weakened during multi-step generation, completely decouples policy improvement from underlying policy representation, and further optimizes continuous control policies. DR-APO combines advantage-guided objective improvement with action fitting adapted to policy representation, from continuous control validation to embodied navigation post-training. Notably, the general training effect of DEEP Robotics' DR-APO even surpasses specialized model training. In five MuJoCo-v5 continuous control benchmark tests, DR-APO ranked first in nine of ten metrics on diffusion policies and first in all ten metrics on flow policies, achieving the best comprehensive performance of a single policy; in embodied visual navigation post-training, the simulation success rate of the pretrained model NavDP increased from 70.33% to 89.55%; in real-scenario tests of DEEP Robotics' Lynx M20 wheeled-legged robot, the navigation success rate increased significantly from 25% to 65%. Embodied agent DR-Agent: The first continuously self-evolving embodied navigation agent. DR-Agent, released by DEEP Robotics in collaboration with the University of California, San Diego and Zhejiang University, is the first self-evolving 4D belief embodied navigation agent oriented toward a continuously evolving world. Aiming at "persistent navigation after invisible object displacement," the team proposed the EvolvingNav method: building a time-indexed predictive 4D belief model, equipped with an event-driven complete filter of "prediction-observation-replanning," enabling the robot to perform temporal reasoning in combination with arrival time and complete target verification relying on vision—it has memory and can continuously optimize behavioral policies based on memory, achieving dynamic iteration and continuous updating of the memory system in human-robot interaction scenarios. DR-Agent overview. Data closed-loop platform DR-Benchmark: 833.6 hours and 3,294.7 kilometers of real operation collection. DEEP Robotics, together with the University of Illinois Urbana-Champaign, the Hong Kong University of Science and Technology, Zhejiang University, Tsinghua University, and MIT, released the DR-Benchmark data closed-loop platform. Its core dataset LynxScenes is the first large-scale dataset for real outdoor scenarios of wheeled-legged robots, filling the data gap in real embodied navigation scenarios and filling the blank in evaluation benchmarks. Overall framework of LynxScenes, covering real robot collection, multimodal observation, spatial and behavioral annotation, and four downstream tasks. Based on DEEP Robotics' Lynx M20 wheeled-legged robot, the team completed a cumulative 833.6 hours and 3,294.7 kilometers of real operation collection, building about 100,000 navigation clips and 3 million annotated frames, providing four types of supervision: 3D objects, road structures, behavioral instructions, and expert demonstration trajectories, covering 27 object categories, 4 road structure types, 3 levels, and 20 navigation instruction categories. LynxScenes is the first to unify the four tasks of instruction understanding, environmental perception, future prediction, and scene reconstruction into the same real data benchmark, and on this basis completed a systematic "checkup" of 25 representative models, redefining the evaluation dimensions of embodied navigation.

Source

Shanghai Observer