Understand this story
Apple published the paper "Amortizing Maximum Inner Product Search with Learned Support Functions" at ICML 2026, proposing to train a small neural network offline for a fixed-key database, treating Maximum Inner Product Search (MIPS) as a support function in convex analysis to learn, thereby routing queries faster and more accurately at inference time. The method includes SupportNet and KeyNet: the former uses an ICNN to learn the support function and can compute it per cluster to improve routing, while the latter directly outputs the best-matching key vector and constrains it with a consistency loss based on Euler's homogeneous function theorem. Both are trained on samples from the relevant query distribution, trading a one-time amortized cost for more efficient RAG retrieval at inference time. Currently, this work is publicly documented based on first-party materials released by Apple Machine Learning Research.
AI-generated from public reports
Developments
1 confirmed developments
Apple publishes paper on amortized MIPS retrieval at ICML 2026
Source:Apple Machine Learning Research
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 6
Apple Machine Learning ResearchFirst-party
Apple Proposes SupportNet and KeyNet: Using Small Neural Networks to Accelerate MIPS Retrieval and Improve RAG Efficiency
In the ICML 2026 paper "Amortizing Maximum Inner Product Search with Learned Support Functions," the Apple research team proposes training a small neural network offline for a fixed key database, treating MIPS as a support function in convex analysis to learn it, thereby routing queries faster and more accurately at inference time. The methods include SupportNet (using an ICNN to learn the support function, which can be computed per cluster to improve routing) and KeyNet (directly outputting the best-matching key vector, constrained by a consistency loss based on the Euler homogeneous function theorem). Both are trained on samples from the relevant query distribution, trading a one-time amortized cost for more efficient RAG retrieval during inference.