Ai2 Releases MolmoMotion, a Language-Guided 3D Motion Prediction Model
Ai2 releases MolmoMotion: by inputting video frames, 3D points on objects, and action descriptions, it can predict the trajectories of these points in 3D space over the next few seconds, outperforming existing prediction methods. It also open-sources the MolmoMotion-1M dataset (derived from 1.16M videos) and the PointMotionBench benchmark (2.7K video clips), and releases the model weights. In simulation, a control policy based on MolmoMotion achieved a 76.3% success rate on pick-and-place tasks, higher than the 56.0% achieved by one based on Molmo 2.
Ai2 releases MolmoMotion: by inputting video frames, 3D points on objects, and action descriptions, it can predict the trajectories of these points in 3D space over the next few seconds, outperforming existing prediction methods. It also open-sources the MolmoMotion-1M dataset (derived from 1.16M videos) and the PointMotionBench benchmark (2.7K video clips), and releases the model weights. In simulation, a control policy based on MolmoMotion achieved a 76.3% success rate on pick-and-place tasks,
Matched fieldsBody


