智源 FlagOpen 开源 Robo-Dopamine：面向高精度机器人操作的通用过程奖励模型（CVPR 2026）

FlagOpen; Robo-Dopamine

Repository navigation

NotificationsYou must be signed in to change notification settings; Fork68 (68); Star660 (660)You must be signed in to star a repository

Repository files navigation

README; Apache-2.0 license

Joy is dopamine’s handiwork—whether in humans or in robotics.

2026-05-13: 🤗 We released Robo-Dopamine-GRM-Dataset, fully 35M training dataset for Robo-Dopamine-GRM series; 2026-04-05: 🤗 We released Robo-Dopamine-GRM-2.0-4B-Preview model in HF. This is a lightweight GRM-2.0 preview version, achieving approximately 80% accuracy of Robo-Dopamine-GRM-2.0-8B-Preview; 2026-03-23: 📊 We employ Robo-Dopamine-GRM as a Judge to evaluate existing VLAs on the RoboChallenge benchmark across multiple dimensions and metrics. Please refer to the 📊 Leaderboard and 📑 Blog for details; 2026-03-05: 🔥🔥🔥 We released Robo-Dopamine-GRM-2.0-8B-Preview model in HF. Highly recommend trying the more versatile and stable GRM-2.0-preview version. It currently supports single-view/multi-view use cases, both with and without reference target images, please refer to Quick Start for details; 2026-03-02: 🤗 We released Robo-Dopamine-GRM-8B model in HF; 2026-02-22: 🔥🔥🔥 Robo-Dopamine gets accepted to CVPR 2026! See you in Denver, Colorado, USA!; 2026-02-10: ⚡ We released data generation pipeline and finetune codes. Try to finetune with your own data; 2026-01-26: 🔍 We released Robo-Dopamine-Bench benchmark and evaluation codes; 2026-01-08: 🤗 We released Robo-Dopamine-GRM-3B model and inference codes; 2025-12-30: ✨ Codes, Dataset and Weights are coming soon! Stay tuned for updates; 2025-12-30: 🔥 We released our Project Page of Robo-Dopamine

🎯 TODO

Release Robo-Dopamine-GRM-3B model and inference codes; Release Robo-Dopamine-Bench benchmark and evaluation codes; Release data generation pipeline and finetune codes; Release Robo-Dopamine-GRM-8B model; Release more powerful and stable Robo-Dopamine-GRM-2.0-8B-Preview model; Release a lightweight Robo-Dopamine-GRM-2.0-4B-Preview model; Release full GRM dataset and GRM pre-training codes

🤖 Overview

Robo-Dopamine is composed of two core components: (a) Dopamine-Reward Modeling Method -- At the heart of our reward modeling is to build the General Reward Model (GRM), a vision-language model that is prompted with a task description and conditioned on multi-view images of initial, goal, "BEFORE," and "AFTER" states to predict a relative progress or regress hop. To ensure a stable and accurate signal, we employ Multi-Perspective Progress Fusion, which combines incremental, forward-anchored, and backward-anchored predictions into a final fused reward. And (b) Dopamine-RL Training Framework -- The Dopamine-RL framework first adapts the pre-trained GRM to a novel task using a single demonstration, i.e., One-Shot GRM Adaptation. Subsequently, it uses a theoretically-sound Policy-Invariant Reward Shaping method to convert the GRM's dense output into a reward signal that accelerates learning without altering the optimal policy. This approach is universally compatible with a wide range of RL algorithms.

The following are simple and practical examples of the three inference modes (Incremental-Mode, Forward-Mode, and Backward-Mode). In practice, to predict the task state reward more accurately, we highly recommend averaging the inference reward results from all three modes to use as the final reward in RL.

✨ More Cases for Testing

Many thanks to Robometer for providing more interesting test examples 🤗. To better demonstrate the usage of 'single-view without goal image' with our latest Robo-Dopamine-GRM-2.0-8B-Preview model, we also provide the following reference script for your easy tests.

We have attached the visualization results from these tests below. Please feel free to open an issue if you have any questions when testing the provided examples or your own test cases. 🤗 🤗 🤗

🔍 Evaluation

0. Download Robo-Dopamine-Bench from huggingface.

1. Evaluate local GRM with vLLM.

2. Evaluate other models with API.

EVALUATION RESULTS

⚡ Fine-Tuning

Step 1. Reconstruct Your Own Dataset

Raw Data Directory Structure: The dataset/example_raw_data directory serves as an EXAMPLE to demonstrate the required structure for your own raw data, ensuring compatibility with our provided data processing scripts.

Step 2. Process Your Own Dataset

Here, we use dataset/example_raw_data as an example.

Step 3. Fine-Tune GRM with Your Own Dataset

Add the meta-info of your own dataset to train/qwenvl/data/__init__.py

Modify the path of training script train/scripts/finetune_grm.sh

Launch the training script

Before you launch training, make sure that you have installed the packages below:

Then, run the script below:

📑 Citation

If you find our work helpful, feel free to cite it: