A systematic review conducted a comparability audit of evaluation protocols for 66 LLM software repair systems

A systematic review conducted a comparability audit of the evaluation protocols of 66 LLM software repair systems. The review classifies the 66 systems into four categories—fine-tuning-based, prompting-based, pipeline-based, and agent-based—according to "where the repair capability lies and who controls the repair loop." The audit shows that among the 66 systems, there are 2,145 pairwise pairings, of which only 1 pair is truly comparable; therefore, "which paradigm is stronger" cannot be answered under the current evidence. The review's corpus time window is from January 2022 to April 2026, in which only 13 systems perform repository-level repair, and 10/66 assume the fault location is known.

AI-generated from public reports

1. Report date Oct 21· 1 reports

   English translation is being prepared.

   Source:English text is being prepared.