Baidu ERNIE discloses RSI progress: ERNIE Evolve has been used in the development of ERNIE 5.5 Flash

On November 9, Baidu's ERNIE disclosed the latest progress on RSI for the first time, unveiling ERNIE Evolve, an automated research system involved in the development of ERNIE 5.5 Flash. It can autonomously propose research directions, conduct experiments, verify results, and accumulate experience around goals and constraints; it can both continuously optimize existing paths and explore new directions based on actual results. On the three evaluation metrics of the two tasks NanoChat AutoResearch and NanoGPT Speedrun, it achieved SOTA performance that has been publicly verified to date. Currently, ERNIE Evolve has been applied to the inference optimization of the soon-to-be-released ERNIE 5.5 Flash, including performance optimization for low-latency scenarios on Kunlun Core P800, deterministic inference throughput optimization, and extreme decoding speed optimization.

ERNIE Evolve benchmark performance

Recursive Self-Improvement (RSI) is one of the important exploration directions in the global AI field today. Its core idea is to let AI directly participate in its own R&D process, continuously optimize its own capabilities, and form a self-improvement loop. OpenAI, Anthropic, Tencent, Zhipu, and many other vendors are all exploring the introduction of RSI into model R&D.

ERNIE Evolve organizes research using a two-layer loop. The outer loop combines historical experimental results and existing knowledge to propose new research directions and arrange multiple research lines for parallel exploration; the inner loop continuously performs code modifications, effect evaluations, and experience accumulation for specific directions. Internally, the system consists of multiple agents working in division of labor: the Director is responsible for selecting research directions, the Selector is responsible for determining the goals and code starting points for each round of experiments, and the Executor is responsible for modifying code and generating new versions. After each round of experiments, an independent evaluation program, the Evaluator, is responsible for evaluating code performance and experimental results, while the Reviewer further checks whether the research has deviated from the goals and whether there are evaluation loopholes. Only improvements that pass evaluation and review can be retained. The code changes, results, and experience of each round of experiments are recorded for reference in subsequent research.

ERNIE Evolve architecture

After integrating ERNIE Evolve into the real inference optimization of ERNIE 5.5 Flash, the system continued to achieve significant gains in the production environment.

· In the optimization of low-latency scenarios on Kunlun Core P800, ERNIE Evolve spent about 168 hours and went through 4 rounds of direction exploration, successively integrating native fused operators and enabling speculative decoding, then optimizing the host scheduling and synchronization mechanisms and the communication path. The generation speed of the optimal version increased to 623.6% of the baseline.

· In deterministic inference throughput optimization, ERNIE Evolve spent 3 days exploring 3 directions around deterministic sampling, deterministic switches, and operator parallel strategies. Under the premise of strictly maintaining determinism, it ultimately increased throughput to 133.7% of the baseline, shortened the average per-token generation time by 26%, and reduced first-token latency by 12%.

· In extreme decoding speed optimization, ERNIE Evolve explored a total of 7 research directions, of which 3 directions brought effective improvements through optimization of MoE and mHC operators, matrix multiplication and scheduling strategies, and the critical path of the draft model, ultimately achieving a cumulative inference performance improvement to 114.9% of the baseline, with accuracy consistent with the baseline.

The above changes have all passed the ERNIE team's stress testing and correctness checks and completed production-level verification.

The ERNIE technical blog points out that inference infrastructure optimization is the first landing point of RSI in real R&D. In the future, the same research loop will gradually extend to training system optimization, model capability research, and evaluation system iteration, driving AI to autonomously discover problems, propose hypotheses, design experiments, and verify results. The expansion process will be carried out within clear goals, permissions, and resource boundaries, and will be subject to continuous safety evaluation and human supervision.

On November 5, Baidu officially open-sourced the latest base model ERNIE 5.5 Flash Base, which achieved SOTA performance among models of the same size on multiple benchmarks including knowledge, reasoning, mathematics, code, multilingual, and long context. A person close to Baidu revealed that the new ERNIE model may be released soon.