RoboCOIN：基于 LeRobot 的机器人数据集与部署工具包

FlagOpen; RoboCOIN

Repository navigation

NotificationsYou must be signed in to change notification settings; Fork14 (14); Star216 (216)You must be signed in to star a repository

RoboCoin + LeRobot integration

Repository files navigation

📄READMEs: English | 中文 | LeRobot

🔥[9 Dec. 2025] Our datasets have reached a total of 300,000 downloads on HuggingFace and ModelScope!; 🔥[24 Nov. 2025] Our technical report is available on ArXiv!

Table of Contents

RoboCOIN

Discover and Download Datasets; Load a Dataset; Lerobot features explanation

observation.state / action; eef_sim_pose_state / eef_sim_pose_action; Upcoming Highlights; Robot Control

Trajectory Replay; Model Inference

Overview

As the official companion toolkit for the [RoboCOIN Dataset], this project is built upon the LeRobot repository. It maintains full compatibility with LeRobot’s data format while adding support for rich metadata—including subtasks, scene descriptions, and motion descriptions. RoboCOIN provides an end-to-end pipeline for dataset discovery, download, and standardized loading, along with model deployment capabilities across multiple robotic platforms.

Key Features:

Dataset Management: Seamless retrieval, downloading, and DataLoader-based loading of datasets, with full support for subtask, scene, and motion annotation metadata; Unified Robot Control Interface: Supports integration with diverse robotic platforms, including SDK-based control (e.g., Piper, Realman) and general-purpose ROS/MoveIt-based control; Standardized Unit Conversion: Built-in utilities for cross-platform unit handling (e.g., degree ↔ radian conversion); Visualization Tools: 2D/3D trajectory plotting and synchronized camera image rendering; Policy Inference & Deployment: Ready-to-use inference pipelines for both LeRobot Policy and OpenPI Policy, enabling direct robot control from trained models

Installation

Dataset Discovery, Download, and Loading

Discover and Download Datasets

Browse available datasets at: [https://flagopen.github.io/RoboCOIN-DataManager/] We will continuously update the datasets. You can find the latest datasets on the page above.

The above GIF shows how to discovery, download, and use RoboCOIN datasets.

Load a Dataset

Lerobot features explanation

observation.state / action

These features represent data collected from the robot arms (slave/master). In the absence of robot action data, actions are derived from the observation.state sequence. The standardized fields are:

eef_sim_pose_state / eef_sim_pose_action

Note: {dir} is a placeholder that stands for left or right.

Upcoming Highlights

Version Compatibility: RoboCOIN currently supports LeRobot v2.1 data format. Support for v3.0 data format is coming soon; Codebase Origin: This project is currently based on LeRobot v0.3.4. Future releases will evolve into a fully compatible LeRobot extension plugin, maintaining seamless interoperability with the official LeRobot repository

Robot Control

The robot control module provides a unified interface for various robotic platforms, supporting SDK-based control (e.g., Piper, Realman) and general-purpose ROS/MoveIt-based control. It includes features such as standardized unit conversion, absolute and relative position control, and trajectory visualization.

Robot Script Structure

All robot scripts are located under src/lerobot/robots. Taking the Realman robot platform as an example, all relevant files are located in src/lerobot/robots/realman(single arm) and src/lerobot/robots/bi_realman(dual arm):

Base Robot Configuration Classes

Inheritance Relationship：

The base configuration for robot platforms is located at src/lerobot/robots/base_robot/configuration_base_robot.py：

Parameter Details：

The dual-arm robot base configuration class is located at src/lerobot/robots/base_robot/configuration_bi_base_robot.py, inheriting from the single-arm base configuration:

Parameter Details：

Specific Robot Configuration Classes

Each specific robot has dedicated configuration inheriting from the robot base configuration. Configure according to the specific robot SDK.

Inheritance relationship, taking Realman as example:

Taking Realman as example, located at src/lerobot/robots/realman/configuration_realman.py：

For dual-arm Realman, configuration class is located at src/lerobot/robots/bi_realman/configuration_bi_realman.py：

Specific Feature Descriptions

Unified Unit Conversion

This module is located at src/lerobot/robots/base_robot/units_transform.py, providing unit conversion functionality for length and angle measurements, supporting unified unit management in robot control systems: length uses meters (m), angles use radians (rad).

Length Unit Conversion: Standard unit is meter (m), supports conversion between micrometer, millimeter, centimeter, meter.

Angle Unit Conversion: Standard unit is radian (rad), supports conversion between millidegree, degree, and radian.

During inference, the control units of the robot platform may differ from the model input/output units. This module provides unified conversion interfaces to ensure unit consistency and correctness during control:

Robot state to model input conversion: Robot specific units -> Standard units -> Model specific units; Model output to robot control conversion: Model specific units -> Standard units -> Robot specific units

Absolute and Relative Position Control

Provides three modes of position control: absolute, relative to previous state, and relative to initial state, applicable to both joint control and end effector control:

Absolute position control (absolute): Directly use model output position as target position; Relative to previous state position control (relative to previous): Use model output position as delta relative to previous state to calculate target position

Without action chunking: Action = Current state + Model output; With action chunking: Action = Current state + All model output chunks, update current state after all executions complete; Relative to initial state position control (relative to initial): Use model output position as delta relative to initial state to calculate target position

Example control flow using action chunking with relative to previous state position control:

Usage Instructions

⚠️ Before running robot control scripts, please read all the configuration items carefully and modify them according to your robot platform SDK and settings.

Trajectory Replay

Robot platform configuration options can be modified in configuration class files or passed via command line. Taking dual-arm Realman as example, command is as follows:

The above command specifies Realman left and right arm IP/ports, and loads head, left hand, right hand cameras. During trajectory replay, control will be based on data in .

Model Inference

LeRobot Policy Based Inference (Naive)

Run inference script directly, taking dual-arm Realman as example, command is as follows:

The above command initializes Realman pose, loads head, left hand, right hand cameras, passes "do something" as prompt, loads local model for inference, and obtains actions to control the robot platform. You can press "q" in console to exit anytime, then press "y/n" to indicate task success/failure. Video will be saved to results/directory.

Run LeRobot Server, see src/lerobot/scripts/server/policy_server.py, command as follows:

The above command starts a service listening on 127.0.0.1:18080.

Run client program, taking dual-arm Realman as example, command as follows:

The above command initializes Realman pose, loads head, left hand, right hand cameras, passes "do something" as prompt, loads ACT model for inference, and obtains actions to control the robot platform.

OpenPI Policy Based Inference

Run OpenPI Server, see OpenPI official repository; Run client program, taking Realman as example, command as follows:

The above command initializes Realman pose, loads head, left hand, right hand cameras, passes "put peach into basket" as prompt, and obtains actions to control the robot platform.

During inference, press "q" in console to exit anytime, then press "y/n" to indicate task success/failure. Video will be saved to results/directory.

First write a configuration class for the current task, e.g. src/lerobot/scripts/server/task_configs/towel_basket.py:

Then run command:

During inference, it starts from the first subtask, press "s" to switch to next subtask.

Press "q" in console to exit anytime, then press "y/n" to indicate task success/failure. Video will be saved to results/ directory.

Adding Custom Robots

Create a new folder under src/lerobot/robots/directory named after your robot, e.g. my_robot; Create the following files in this folder:

__init__.py: Initialization file; my_robot.py: Implement robot control logic; configuration_my_robot.py: Define robot configuration class, inheriting from RobotConfig; Define robot configuration in configuration_my_robot.py, including SDK-specific configuration and required base configuration parameters; Implement robot control logic in my_robot.py, inheriting from BaseRobot; Implement all abstract methods:

_check_dependencys(self): Check robot dependencies; _connect_arm(self): Connect to robot; _disconnect_arm(self): Disconnect from robot; _set_joint_state(self, joint_state: np.ndarray): Set robot joint state, input is joint state numpy array, units as defined in configuration class joint_units; _get_joint_state(self) -> np.ndarray: Get current robot joint state, returns joint state numpy array, units as defined in configuration class joint_units; _set_ee_state(self, ee_state: np.ndarray): Set robot end effector state, input is end effector state numpy array, units as defined in configuration class pose_units; _get_ee_state(self) -> np.ndarray: Get current robot end effector state, returns end effector state numpy array, units as defined in configuration class pose_units; Refer to other robot implementation classes, implement other control modes (optional):

my_robot_end_effector.py: Implement end effector based control logic, inheriting from BaseRobotEndEffectorand my_robot.py; bi_my_robot.py: Implement dual-arm robot control logic, inheriting from BiBaseRobotand my_robot.py; bi_my_robot_end_effector.py: Implement dual-arm robot end effector based control logic, inheriting from BiBaseRobotEndEffectorand my_robot_end_effector.py; Register your robot configuration class in src/lerobot/robots/utils.py:; Import your robot implementation class at the beginning of inference scripts:; Now you can use your custom robot via command line parameter --robot.type=my_robot

Robot Teleoperation

The teleoperator module provides teleoperation functionality for various robot platforms, allowing users to control robots through teleoperation devices. It provides various teleoperation methods, including code-based and hardware-based teleoperation.

Teleoperator Script Structure

All teleoperator scripts are located under src/lerobot/teleoperators. Taking the Realman teleoperator as an example, all relevant files are located in src/lerobot/teleoperators/realman_leader:

Base Teleoperator Configuration Classes

Inheritance Relationship：

The base configuration for teleoperators is located at src/lerobot/teleoperators/base_teleoperator/config_base_teleoperator.py:

Parameter Details：

The dual-arm robot teleoperator base configuration class is located at src/lerobot/teleoperators/bi_base_leader/config_bi_base_leader.py, inheriting from the single-arm base configuration:

Specific Teleoperator Configuration Classes

Each specific teleoperator has dedicated configuration inheriting from the teleoperator base configuration. Configure according to the specific robot SDK.

Inheritance relationship, taking Realman teleoperator as example:

Taking Realman teleoperator as example, located at src/lerobot/teleoperators/realman_leader/config_realman_leader.py：

For dual-arm Realman teleoperator, configuration class is located at src/lerobot/teleoperators/bi_realman_leader/config_bi_realman_leader.py：

Usage Instructions

⚠️ Before running robot teleoperation scripts, please read all the configuration items carefully and modify them according to your robot platform SDK and settings.

Connect the leader & follower arm hardware device to the computer; Run the teleoperation recording script, taking Piper robot as example, command is as follows:

The above command connects to Piper follower robot via CAN bus can0, loads front camera, connects to Piper leader robot via CAN bus can1, and records 3 episodes of teleoperation data with the task "do something" into .

You can press direction key (->) to start next episode, press direction key ( .

Adding Custom Teleoperators

Create a new folder under src/lerobot/teleoperators/ directory named after your teleoperator, e.g. my_robot_leader; Create the following files in this folder:

__init__.py: Initialization file; my_robot_leader.py: Implement teleoperator control logic; configuration_my_robot_leader.py: Define teleoperator configuration class, inheriting from TeleoperatorConfig; my_robot_leader_end_effector.py (optional): Implement end effector based teleoperator control logic; Define teleoperator configuration in configuration_my_robot_leader.py, including SDK-specific configuration and required base configuration parameters; Implement teleoperator control logic in my_robot_leader.py, inheriting from BaseLeader; Implement all abstract methods:

_check_dependencys(self): Check teleoperator dependencies; _connect_leader(self): Connect to leader hardware device; _disconnect_leader(self): Disconnect from leader hardware device; _get_joint_state(self) -> np.ndarray: Get current leader joint state, returns joint state numpy array, units as defined in configuration class joint_units; _get_ee_state(self) -> np.ndarray: Get current leader end effector state, returns end effector state numpy array, units as defined in configuration class pose_units; Refer to other teleoperator implementation classes, implement other control modes (optional):

my_robot_leader_end_effector.py: Implement end effector based teleoperator control logic, inheriting from BaseLeaderEndEffector and my_robot_leader.py; bi_my_robot_leader.py: Implement dual-arm teleoperator control logic, inheriting from BiBaseLeader and my_robot_leader.py; bi_my_robot_leader_end_effector.py: Implement dual-arm end effector based teleoperator control logic, inheriting from BiBaseLeaderEndEffector and my_robot_leader_end_effector.py; Register your teleoperator configuration class in src/lerobot/teleoperators/utils.py:; Import your teleoperator implementation class at the beginning of recording scripts:; Now you can use your custom teleoperator via command line parameter --teleop.type=my_robot_leader

Acknowledgements

Thanks to the following open-source projects for their support and assistance to RoboCOIN:

LeRobot; OpenPI

Welcome to scan the QR code to join the official RoboCOIN WeChat group for discussion.

RoboCoin + LeRobot integration