trendingAI & ML — INTERMEDIATE
Vector Databases for Engineers: Index Types & Metadata Filtering
“Find the closest vectors” hides two real engineering decisions: which index structure to build, and how to combine that search with the ordinary metadata filters every production query actually needs.
The Quick Answer
Vector databases pick from a small set of approximate nearest neighbor (ANN) index types — most commonly HNSW, IVF, or LSH — each trading off speed, recall, and memory differently. Production queries almost always also carry a metadata filter (“only in-stock items,” “only documents from this user”), and whether that filter runs before or after the similarity search meaningfully changes both correctness and performance.
The Three Common Index Types
| Index | How it narrows the search | Trade-off |
|---|---|---|
| HNSW | Multi-layer graph — jump via long-range links at the top, refine locally at the bottom. | High recall, low latency; heavier memory footprint. |
| IVF | Pre-clusters vectors (e.g. via k-means), searches only the closest clusters. | Smaller memory footprint, tunable via how many clusters get scanned; somewhat lower recall than HNSW at the same speed. |
| LSH | Hashes vectors so similar ones fall in the same bucket; only compares within a query’s bucket. | Simple, no training step; needs many hash tables for strong recall, and degrades as dimensions grow. |
A practical default: reach for HNSW when memory is available and recall matters most, IVF (often paired with quantization) when the corpus is huge and memory is the binding constraint, and LSH mainly when you need a simple index with no training phase.
Metadata Filtering: Pre- vs. Post-Filter
Real queries are rarely just “find similar vectors” — they’re “find similar vectors where category = electronics” or where user_id = this user. There are two ways to combine that filter with the similarity search:
- Pre-filtering — narrow to matching documents first, then run ANN search only on that smaller set. Cheap when the filter is highly selective.
- Post-filtering — run ANN search first to get the closest candidates, then discard the ones that fail the filter. Simple and works with any index, but risks returning too few results if the filter is narrow and most of the initial candidates get discarded.
Try both strategies below against the same 1-million-document collection, under a narrow filter and a broad one.
Hybrid Search
Vector search is excellent at meaning but can miss an exact term that genuinely matters — a product SKU, a specific error code, a proper noun the embedding model never learned to weight heavily. Hybrid search runs a vector search and a traditional keyword search side by side, then merges both result sets — combining the meaning-matching strength from embeddings with the precision of exact keyword matching, rather than picking one at the expense of the other.
Fun Fact
IVF’s clustering step usually runs k-means once, up front, to decide the cluster boundaries — meaning the quality of every future search depends on how well that initial clustering pass captured the actual shape of your data. A poorly-clustered index can search fast and still miss the right answer, illustrating why speed and correctness are genuinely separate concerns here, not the same thing measured two ways.
Test Yourself
What is the main trade-off HNSW makes compared to IVF?
How does IVF (Inverted File Index) narrow down a search?
When does pre-filtering (filter, then search) work best?
What risk does post-filtering (search, then filter) carry with a narrow filter?
What does hybrid search combine?
Ready for a worked compression example with real numbers, plus actual metadata filtering code and why getting it wrong can silently drop the best match? Continue to the Advanced level →