Tsinghua, AgiBot, and Infinigence AI propose Collective Experience Learning (CEL); 96 robots co-evolve and surpass human success rates

Every execution of a robot in the real world generates experience. How to make use of this experience has always been a difficult problem. The traditional approach relies on manual correction, with teleoperators monitoring robot execution and taking over to provide correct actions when the policy errs. This model can still be sustained with a few robots, but as the fleet scales up, human supervision will sooner or later fail to keep pace with growing demand.

To address this problem, a joint team from Tsinghua University, AgiBot, and Infinigence AI proposed Collective Experience Learning (CEL), a learning paradigm that enables deployed robots to share execution experience and jointly improve the same policy.

The algorithmic implementation of CEL is Failure-Guided Experience Retrieval Fine-Tuning (FERT): it uses the context of a robot's execution failure to retrieve relevant trajectory segments from successful experiences shared across the entire fleet, and uses the actions recorded in them as supervision signals to update the policy. The entire process requires no additional collection of expert demonstrations or corrective actions. Humans only need to label each execution as success or failure and reset the environment; the actions themselves are all provided by the robots.

Figure | Three sources of experience for robot policy learning: human demonstrations, expert corrections, and collective fleet experience

To support continuous learning on large-scale fleets, the team also built a Fault-Tolerant Asynchronous Training System (FATS). In this way, even if individual robots in the fleet are interrupted due to failures, resets, or maintenance, experience collection, policy optimization, and policy distribution can continue.

On five real-world manipulation tasks, including phone packaging, flower arrangement, and clothes folding, the research team conducted 120 trials for each task. The robots involved in evaluation did not participate in training data collection, and the final policies achieved success rates of 93.3% to 99.2%. When the collection time per robot was fixed and only the number of robots was increased, the larger the fleet and the more experience, the better FERT performed overall. In the multi-stage phone packaging task, after learning with 96 robots, the task success rate rose from 79.2% to 98.3%.

These results show that collective experience can become a feasible foundation for scalable robot learning: as deployment scale expands, the supply of experience grows in tandem, without requiring expert supervision to expand proportionally. This provides a path for robots to continuously improve policy reliability during deployment.

I. Research Methods

The algorithm and system implementations of CEL are FERT and FATS, respectively.

The core idea of FERT can be summarized in one sentence: success provides the object to imitate, and failure determines what to learn. The specific process is divided into the following four steps:

Step one, asynchronous experience collection. Robots in the fleet continuously execute tasks autonomously, and each completed trajectory is labeled with a binary success or failure outcome. Successful trajectories enter the collective experience buffer as learning material; failed trajectories do not provide actions and are used only as retrieval cues. This ensures that the system can learn from failures without imitating failed actions.

Step two, experience encoding. The output of a frozen VLA visual encoder (after spatial pooling) and the proprioceptive joint states are separately normalized and then concatenated to form a unified retrieval representation, enabling experience from different robots to be compared with one another.

Step three, failure-guided retrieval, which is also the key to the method. A binary label can only indicate that a trajectory failed, but cannot locate where the failure occurred. FERT takes the observation sequence from about 3 seconds before the failure endpoint as a query and uses subsequence dynamic time warping with slope constraints (S-DTW) to find temporally aligned, state-similar successful segments in the successful experience library. Intuitively, this answers "in a similar situation, what should be done to succeed?"

Step four, priority fine-tuning. Retrieved matching segments receive higher sampling priority, and repeatedly occurring failure situations cause the relevant successful behaviors to be learned more frequently; at the same time, the priority decays exponentially over time, ensuring that learning always focuses on the parts where the policy is currently most prone to failure. Policy updates use log-likelihood fitting of action chunks and require no value function, critic, or advantage estimation. Theoretically, the team proved that under conditions of successful replay and retrieval alignment, the update direction of FERT is consistent with the goal of improving task success rate, and retrieval weighting brings a first-order positive contribution.

Figure | Overall pipeline of FERT

Running this learning loop of FERT on a real fleet is no less an engineering challenge than the algorithm itself. The team pointed out two key characteristics: first, the timing requirements for interaction and experience differ. Robots must receive inference results before their next action (millisecond level), while training requires complete episode records (GB level, but they do not need to arrive immediately). Second, physical robots may be interrupted at any time. In actual operation with 96 robots, on average one robot failed every 22 minutes.

Figure | FATS architecture

To solve these two challenges, FATS adopts a dual-plane architecture.

Among them, the data plane SwarmLink connects robot execution to shared cloud inference and training. Inference requests flow into a common request pool, and paused robots automatically yield compute; episode data is streamed upload during execution, and observations already used for inference are reused directly in the cloud to avoid duplicate transmission; interactive traffic takes priority over batch training data.

Control plane HotSwarm models the cluster as a dynamic membership: "stations," i.e., fixed locations bound to nodes, devices, and robots, are separated in lifecycle from "workers," i.e., the processes occupying the stations; stations persist throughout training, while workers can be replaced at any time. When a robot fails, the system restarts its worker in place, re-establishes the connection with a new version number, and replays the interrupted calls, while the other robots are unaffected; stale actions computed for a failed worker are identified and discarded, preventing data contamination from action-observation mismatches from polluting training. Recovery strategies are also designed in tiers: environment workers restart in place at any time, at a cost of at most one episode; training workers periodically checkpoint and recover from the latest checkpoint, preserving optimizer state accumulated over hours.

2. Experimental Results

The experiments covered five real tabletop manipulation tasks: flower arrangement (flexible objects), clothes folding (deformable fabric), phone charging (plugging and precise alignment), stool assembly (small objects and tool use), and phone packaging (multi-stage placement, taking 2-3 minutes). The hardware was the dual-arm AgiBot G2 robot, and the base model was π0.

At the maximum cluster size for each task, multi-robot HG-DAgger raised the five-task average success rate from 36.3% with SFT to 81.0%; FERT, without using any additional expert data, further raised the average to 97.3%, reducing the average failure rate by 86.0% relative. Among these, phone charging rose from 60.0% to 97.5%, a relative failure-rate reduction of 93.8%; phone packaging rose from 79.2% to 98.3%.

Figure | Main results on the five tasks

FERT surpassed human teleoperation on all five tasks, averaging 16.7 percentage points higher, and 25.8 percentage points higher on phone charging; it also led both baselines across the board—single-robot online reinforcement learning RLT (12.3 percentage points higher on average) and offline reinforcement learning RECAP, which likewise uses cluster data but retains expert corrections (11.5 percentage points higher on average).

When the collection time per robot was fixed and only the number of robots was increased, FERT's success rate generally continued to rise across the five tasks: in the phone packaging task, as the training cluster expanded from 16 to 96 robots, the success rate rose from 86.7% to 98.3%. In contrast, HG-DAgger, which relies on human intervention, saturated or even declined as the cluster expanded, with limited human monitoring and intervention capacity constituting a performance ceiling. FERT's experience supply, however, grows naturally with cluster size, making the scaling curve of "the more robots, the better the learning" hold. Note that under small clusters FERT may also fall below the initial policy, because successful experience is not yet sufficient to cover the various failure situations the policy encounters.

Figure | Scaling with cluster size

Finally, the research team tested the contribution of each design choice through ablation experiments. Ablations on the phone packaging task (16 training robots, 86.7% success rate in the full configuration) verified the role of each design. Removing failure-guided retrieval dropped the success rate to 64.2%, a decrease of 22.5 percentage points, the largest drop among all ablation items, directly validating the value of "selecting experience by failure situation"; removing the action-context extension before and after the retrieved segments lowered the success rate to 57.5%, showing that the action context surrounding matched segments is equally important for learning successful behavior. The policy synchronization frequency was optimal at every 500 optimizer steps: too sparse delays use of the new policy, while too dense leaves insufficient experience accumulated under each policy version; the priority retention factor was optimal at a medium value of 0.88, reflecting the trade-off between retaining historical retrieval information and responding to the latest failures.

Figure | Ablation experiment results

At the system level, 96 robots collected 9,834 episodes over nearly 8 hours of operation, during which 30 failures caused only 1.3% loss of robot time. By contrast, with synchronous collection, the collected volume would be only 33% of the measured amount; and if the whole machine were restarted for 5 minutes after each failure, only 86% could be preserved. The value of the fault-tolerant asynchronous design is thus quantified.

The research team also noted that the evaluation was limited to five task-specific policies on a single platform (AgiBot G2). Broader task distributions, heterogeneous robot platforms, and adaptation to environmental changes over longer operating periods are left for future work.

Overall, CEL brings a new deployment logic. A robot cluster contributes not only task execution but also experience that the entire cluster can learn from and improve on. Through a shared policy, a single robot's experience can be converted into the capability of the whole cluster. As deployment scale expands, the sources of experience grow in tandem. Thus, more robots means not only greater capacity but also better learning, and it is not constrained by the supply of expert supervision.

For more technical details, see the original paper.

This article is from the WeChat public account "Academic Headlines" (ID: SciTouTiao), author: Zhang Jiayi, published by 36Kr with authorization.