Cerebras 如何扩展 OpenHands 用于代码修复智能体的强化学习训练

Nov 24 2025

Training effective code-repair agents requires more than just large language models—it demands a robust infrastructure that can efficiently interact with real-world codebases at scale. After building a comprehensive dataset of real-world issue-resolution instances packaged as Docker images, we faced a critical challenge: existing agentic frameworks like OpenHands were tightly coupled to benchmark-specific assumptions that prevented their use in large-scale reinforcement learning (RL) training loops.

This post details our technical journey in transforming OpenHands from a SWE-Bench evaluation tool into a general-purpose RL training platform capable of handling thousands of diverse Python repositories.

OpenHands provides a sophisticated agentic scaffold through its CodeActAgent, but its evaluation pipeline was architected specifically for SWE-Bench Verified. This tight coupling manifested in several ways:

Our dataset, spanning thousands of Python repositories scraped from PyPI and GitHub, presented fundamentally different requirements. Each repository brought its own:

We completely refactored the evaluation workflow to eliminate dependency on SWE-Bench Verified specific assumptions. The new architecture:

3. Executes tests in containerized environments with full isolation

4. Reports structured pass/fail outcomes compatible with RL reward signals

This transformation effectively converts OpenHands into a general-purpose evaluation engine for code-repair tasks, capable of operating on any repository with minimal configuration

The original OpenHands design separated patch generation and evaluation into independent workflows. Each step would:

For moderately-sized Docker images, this duplication introduced approximately 3 minutes of overhead per instance —completely untenable for large-scale RL training where we need to generate thousands of rollouts per training iteration.

Our solution implements a unified workflow that maintains a persistent runtime context throughout the entire agent trajectory:

Key architectural changes:

This architectural change delivers dramatic efficiency improvements:

Our extended OpenHands fork now provides all essential components for training code-repair agents via RL:

Agents can engage in extended debugging sessions, making multiple edits and receiving feedback at each step—critical for learning complex repair strategies.

Quicker rollout times enable rapid iteration during training, supporting techniques like:

Binary pass/fail signals from test suites provide clear learning objectives while avoiding reward shaping biases.

Detailed execution traces capture:

This rich data supports offline analysis, curriculum learning, and reward model training.

By combining our curated dataset of real-world issues with this extended OpenHands infrastructure, we've established a complete pipeline for training code-repair agents at scale. Current experiments focus on:

Early results indicate that RL-trained models significantly outperform supervised fine-tuning baselines on multi-step repair tasks, particularly on issues requiring iterative debugging and test-driven refinement.

Our work delivers three key technical contributions:

These advances transform OpenHands from a specialized benchmark tool into a scalable platform for training next-generation code-repair agents on real-world software engineering tasks.

This work builds on the excellent foundation provided by the OpenHands team. Our modifications are designed to complement and extend their framework while maintaining compatibility with the original codebase.

Interested in the technical details? Reach out to discuss collaboration opportunities in agentic code generation and reinforcement learning for software engineering.

Performance comparisons are based on third-party benchmarking or internal testing. Observed inference speed improvements versus GPU-based systems may vary depending on workload, configuration, date and models being tested.

1237 E. Arques Ave Sunnyvale, CA 94085