Skip to content
Source: Apple Machine Learning Research

🔍Researchers Train Neural Nets to Replace Brute-Force Vector Search for Retrieval

Your vector search could skip most of the work

TL;DR

Researchers trained small neural networks, SupportNet and KeyNet, to predict the best-matching key for a query without scanning millions of vectors. Because they learn from the real query distribution, they can beat centroid routing on accuracy at inference time. If you run RAG or recommendations on a stable user base, this is worth a look.

Here's the pitch: instead of brute-forcing maximum inner product search across millions of key vectors, train a small neural network to answer the query for you. Two variants came out of this work. SupportNet learns the support function directly using a loosely constrained input-convex neural network, and KeyNet outputs the best-matching key vector outright. Why you should care: most MIPS systems in production, including the retrieval layer in RAG pipelines, use centroid routing. You cluster the keys into k groups, pick the closest centroid, then scan roughly n/k keys. The catch is that a centroid can't represent the spread of its cluster, so routing mistakes happen. SupportNet asks a sharper question: how good is the best key inside this cluster for this query? That's exact routing instead of a coarse guess. The tradeoff is upfront cost. You precompute a large set of sampled queries paired with ground-truth MIPS outputs, then regress against them. A consistency loss based on Euler's theorem for homogeneous functions keeps KeyNet's predictions tied to the underlying support function. Hot take: this only pays off if your query distribution is stable. If your users change behavior weekly, you're retraining constantly.

Researchers Train Neural Nets to Replace Brute-Force Vector Search for Retrieval — Apple Machine Learning Research

Key Points

1

SupportNet learns the support function of the key set using a loosely constrained input-convex neural network, giving exact routing decisions instead of coarse centroid comparisons across clusters.

Keep reading — it's free

Enter your email to keep reading — plus the best of AI & tech, daily. Free, forever.

Also get
or

Already a member? Sign in

MIPSRAGvector searchneural networksretrieval

Comments

Subscribe to join the conversation...

Be the first to comment