ICML 2026 | QiMeng-LibBench: 85 real operators show how far AI still is from migrating libraries across chips

The ICML 2026 paper proposes QiMeng-LibBench (referred to as CLAM-Bench during review), which uses 85 real operators from OpenCV, ncnn, and libjpeg to evaluate the cross-chip library migration capability of AI agents. The 12 "agent + large model" combinations on average got only 20.88% of tasks right on the first attempt, with the best combination at 37.65%; after adding the requirement of no slowdown, almost all of them dropped close to 0. The paper points out that the root cause is that agents only imitate the surface characters of source code rather than understanding the semantics of vector instructions, and proposes a multi-agent framework FSCM that first clarifies the library dependency structure before generating code to improve correctness.