Vector DB Fundamentals

Exact search on a million vectors would take minutes. ANN finds the answer in milliseconds — here's the honest trade.

▶ Watch this reel

What you'll learn

What & why
Why not a plain SQL table
ANN algorithms
Recall vs speed
Metadata filtering
Hybrid search

Remember this

Vector DB = store + k-NN search; B-trees can't index angles, hence ANN
HNSW: multi-layer neighbor graph; tune M/ef, measure recall on YOUR golden set
Filters pre/post trade correctness vs speed; hybrid = BM25 + ANN fused with RRF, optional rerank

What & why

Store vectors + payloads; answer k-NN ('top-k most similar') fast at scale.

Why not a plain SQL table

B-trees index scalar order; similarity-by-angle has none → brute-force scan.
Exact scan is the correctness baseline; ANN is how you beat physics.

ANN algorithms

HNSW (default): multi-layer small-world graph; long jumps → local walk.

- M = links/node, ef = search width → recall vs memory/insert speed.

IVF: cluster first, search promising clusters.
PQ: compress vectors (~10x memory) at some recall.

Recall vs speed

Recall < 100% is the price; it's a budget you set.
Measure hit@5/recall vs brute force on YOUR golden set; tune to SLA.
Backstop: ANN top-50 → exact/cross-encoder re-score (near-exact at low cost).

Metadata filtering

Pre-filter: correct, slow at high selectivity. Post-filter: fast, can starve k.
Ultra-selective always-on filters (tenant) → per-tenant collections.

Hybrid search

BM25 + ANN → Reciprocal Rank Fusion: Σ 1/(k+rank), re-sort.
Optional cross-encoder rerank on the fused top (GA-15).

Code: Hybrid search with Qdrant-style semantics + RRF

import numpy as np

# --- setup (any ANN store: Qdrant/Chroma/pgvector all share this shape)
# collection.upsert(ids, vectors=vecs, payloads=[{text, tenant, year}...])

def vector_search(query_vec, k=50):
    return ann_index.query(query_vec, top_k=k)      # your ANN leg

def keyword_search(query, k=50):
    return bm25_index.search(query, top_k=k)        # your keyword leg

def rrf_fuse(rankings, k=60):
    """Reciprocal Rank Fusion — robust to different score scales."""
    scores = {}
    for ranking in rankings:
        for rank, doc_id in enumerate(ranking):
            scores[doc_id] = scores.get(doc_id, 0) + 1.0 / (k + rank + 1)
    return sorted(scores, key=scores.get, reverse=True)

# --- one hybrid query ---------------------------------------------
vec_hits = vector_search(embed(query), k=50)
key_hits = keyword_search(query, k=50)
final = rrf_fuse([vec_hits, key_hits])[:10]

# --- recall check against brute force (your golden queries) --------
def recall_at5(ann_hits, exact_hits):
    return len(set(ann_hits[:5]) & set(exact_hits[:5])) / 5

# exact: sims = corpus_u @ query_u; top = argsort(-sims)[:k]
# assert recall_at5(ann, exact) >= 0.99  → tune M/ef until this passes