SuperEMeans Combines SuperKMeans and e-means to Speed Up High-Dimensional Vector Clustering by 3×
SuperEMeans combines SuperKMeans with e-means, accelerating high-dimensional vector clustering by up to 3× without reducing clustering quality. The code is open-sourced at https://github.com/cwida/SuperKMeans. Tests on OpenAI and Cohere embedding datasets show speedups over e-means of up to 1.8× at 2 epochs and 3× to 3.2× at 5 epochs. The tests also found that cluster imbalance in e-means increases the number of vectors scanned; raising the revival threshold from 0.1 to 0.3 brings scanning costs on the Cohere dataset closer to those of Lloyd’s k-means.
Why read this
AI score 56
The tests present clustering time, recall, and the number of vectors scanned, revealing that WCSS alone is insufficient for measuring cluster balance and query costs in vector search.
AI Trends