Berkeley team proposes Evolving Agent Teams, using dynamically reorganized agent teams to break through GPU…

GPU Kernel optimization is not simply about making a piece of runnable code faster. When facing Kernels in real production environments, a coding agent needs to repeatedly explore across hardware instructions, memory layouts, parallelization strategies, and the overall architecture. Although it can make progress in the early stages, it will gradually shift toward approaches that are easier to implement and have a lower performance ceiling, ultimately falling into a local optimum.

To address this problem, a team at the University of California, Berkeley (UC Berkeley) proposed "EvolvingAgent Teams." They argue that rather than increasing the number of agents, a central supervisor should, based on the current bottleneck, reorganize the agent team round by round, adjusting role division, collaboration methods, and knowledge management strategies.

The entire Evolving Agent Teams framework is divided into two layers: the lower-layer agent team is responsible for writing, compiling, testing, and analyzing the Kernel; the upper-layer supervisor observes optimization progress, analyzes the reasons for success and failure, and decides how to adjust in the next round.

Figure | Evolving agentTeams framework

If the agent team encounters a bottleneck during debugging, the supervisor can introduce a Debugger; if there is a lack of performance data as a basis for optimization, a Profiling Scientist is added; if key findings from early experiments are not passed on to subsequent agent teams, a Knowledge Integrator is added, responsible for organizing experiment records and action plans. At the same time, the system also maintains a cross-team persistent file system for storing code, role records, failed attempts, hardware observations, and validated strategies.

The MLA paged decode Kernel demonstrates how this mechanism helps the agent team break through the local optimum. MLA is the attention structure adopted by DeepSeek-V3, and its optimization difficulties include high register pressure and the coordination between CUDA Cores and Tensor Cores. In the early stage of the experiment, a single agent tried the WGMMA path multiple times without success, and then switched to the easier-to-debug WMMA. Although WMMA could eventually run, it also limited Tensor Core utilization.

To prevent the agent team from giving up too early, the supervisor added a Disciplinarian, requiring the agent team to retain the WGMMA constraint and make only one modification at a time that could be clearly evaluated. After that, the task was assigned to a Debugger and the main optimization agent running in parallel: the former verified the difficult-to-implement matrix multiplication path, while the latter continued to advance other optimization work.

Figure | The long-running optimization process of the MLA decode Kernel using Claude Opus 4.6

When the supervisor found that key findings from early experiments had not been passed on to subsequent agent teams, it introduced a Knowledge Integrator to establish cross-team experience documentation. This ultimately helped the agent team validate the P@V WGMMA path. After completing the core architecture adjustment, the supervisor also required re-profiling to further optimize performance.

Across four CUDA Kernel tasks, Evolving agent Teams all surpassed strong coding agent baselines. These tasks cover NVIDIA B200 and H100 GPUs, including FlashAttention-4, Gated Delta Net prefill, MLA decode, and GEMM and AllReduce fusion.

Figure | Performance comparison of the four Kernel tasks

Under the same Token budget, in the MLA decode task, the performance of Claude Code was about 51% of the production-environment FlashInfer Kernel, while Evolving Agent Teams reached 74%. By comparison, a fixed four-role team was 31%, and a single Codex agent was 28%. This shows that the performance gap not only comes from the underlying model capability, but also depends on whether the agent team can adjust its organizational form as the task stage changes.

Figure | MLA decoding ablation experiment

After investing more inference budget, the MLA Kernel generated by Evolving Agent Teams achieved an average performance of 1.21 times that of FlashInfer on 47 production workloads in FlashInfer-Bench, among which 36 workloads exceeded the production-environment FlashInfer Kernel, while the remaining 11 workloads were between 0.86 and 0.99 times that of FlashInfer, meaning that there is still a gap on some workloads.

As the runtime increased, the agent team achieved an average speedup of 4.06 times on Gated Delta Net prefill. In the GEMM and AllReduce fusion task, they progressively advanced asynchronous loading, Warp Specialization, and NVLink multicast, ultimately reaching 1.56 times the baseline performance, slightly higher than the 1.54 times reported by ThunderKittens.

Figure | Evolution trajectories of GDN and G+AR under additional budget

In the Flash-KMeans experiment, the agent team rewrote an expert-optimized Triton Kernel into a CUDA WGMMA Kernel, reducing latency by up to about 1.5 times in compute-bound configurations, while in memory-bound configurations the results were close to those of Triton. This shows that the framework also has the ability to further explore and improve on the basis of existing expert-optimized implementations.

In other scenarios, the agent team also achieved good results: on AMD MI350X, it optimized the performance of Kimi Delta Attention prefill to 3.01 times that of the vLLM production HIP Kernel; on NVIDIA B200, it reached 3.44 times that of FlashKDA; on Apple M4 Pro, the MoE expert offloading service enabled Qwen3.8-Flash-Next to reach 30.3 Tokens per second in a 4-bit MLX environment; in the RoboDojo robotic manipulation task, the system scored 62.1 on the held-out seeds, higher than the leaderboard's highest score of 55.3 as of October 26.

Figure | Complete run results under the other four settings

Unlike simply increasing inference budget, these results point in another direction: for long-horizon, open-ended code optimization tasks, the bottleneck is not only whether the agent can propose better modifications, but also whether the current agent team has roles, collaboration protocols, and experience structures suitable for this stage. Evolving Agent Teams attempts to adjust the search process itself.

However, this research still has some limitations. Each of the four main experiments was run twice, and most other scenarios were tested only once, which is not yet sufficient to support adequate statistical comparison. The team said that future work should also include Pareto analysis between cost and performance, as well as controlled studies in noisier domains. Therefore, the current results are better understood as a systematic design exploration of long-horizon agent optimization tasks, rather than universally validated laws of agent team collaboration.

This article is from the WeChat public account "Academic Headlines" (ID: SciTouTiao), author: Academic Headlines, published by 36Kr with authorization.