Anthropic tests language model performance on robotic tasks

Anthropic released research on July 9, 2026, testing the performance of multiple language models on classic control, locomotion navigation, and robotic arm manipulation tasks, and built the Embody benchmark suite. The research results show that model capability is highly dependent on the connection method with the robot: when directly driving joints, most fail, while when using a supervised pretrained controller or a simple orientation tool, the models can complete real navigation and manipulation tasks. In LIBERO direct manipulation, the full task success rate is only 0 to 5.5%; after connecting to a VLA policy, the success rate of each model increases substantially, but the strongest model still falls short of MolmoAct running alone. There is currently no follow-up progress.

AI-generated from public reports

1. Jul 9· 1 reports

   Anthropic tests language model performance on robotic tasks

   Source:Anthropic：The Institute