Sakana AI 在 ICML 2026 提出 Bridging Spherical Black-Box Optimizers
Bridging Spherical Black-Box Optimizers
Sakana AI 在 ICML 2026 发表研究“Bridging Spherical Black-Box Optimizers”，证明参数化方法（如 ES）与非参数化方法（如 CBO）实为同一更新方程的变体，从而可定制混合优化器。团队据此开发 AdaPol 与 SchedPol，将 LLM 合并视为多模态问题，在小数据集上找到多个不同最优解，以更低算力获得泛化性更好的合并模型。
正文
当前显示原文，中文全文译文尚未就绪。
We are pleased to present our research at ICML 2026, “Bridging Spherical Black-Box Optimizers”.
When optimizing through simulators, external APIs, or in reinforcement learning, gradients are often unavailable. Black-Box Optimization (BBO) fills this gap, but the field has been historically split into two categories:
Parametric Methods: Algorithms like Evolution Strategies (ES) scale to high dimensions but only find a single solution; Nonparametric Methods: Algorithms like Consensus-Based Optimization (CBO) find multiple solutions but fail in high dimensions
Our team asked a simple question: what if they are all doing the same thing?
In our paper, we showed that these distinct families are actually variations of a single update equation. By bridging this theoretical gap, we can now engineer custom hybrid optimizers for specific tasks.
A key application of this is merging foundation models. Building on our previous work in Evolutionary Model Merging, we faced a computational challenge. Evaluating large language models at every step is resource-intensive, but using a smaller evaluation dataset causes standard unimodal optimizers to overfit.
By treating LLM merging as a multimodal problem and deploying our newly developed hybrid optimizers, AdaPol and SchedPol, we successfully navigated this issue. The algorithms identified multiple distinct optima on the smaller dataset, allowing us to find generalized, high-quality merges at a fraction of the compute cost.