Section 6 of 8

Embeddings and Retrieval

Embeddings, vector databases, sparse and hybrid retrieval, chunking, reranking, and the search lineage from BoW to E5.

7 items in this section.

Long reads

Embeddings: How Geometry Pretends to Be Meaning

Embeddings aren't an encoding of text — they're an attempt to make geometry behave as if it carried meaning. What it means to compress text into a fixed-length vector, how contrastive learning turns statistical structure into distances and directions, why cosine similarity works (and when it stops), how dimension, chunking, context window, and reranking change the physics of a retrieval pipeline, and where embeddings lie usefully.

embeddings

Read

Updated Jun 8, 2026

How Text Became Geometry

Sixty years of incremental work behind the modern embedding vector. From bag-of-words and BM25, through Word2Vec's distributional hypothesis, to contextual embeddings and contrastive retrieval models. The point isn't trivia — it's that each older idea is still in production today, and the picture only makes sense once you've seen the road that led here.

embeddings
history
retrieval

Read

Updated May 21, 2026

Lab inside

Chunking Strategies

Chunking turns out to be more architectural than it looks. A walk through what a chunk has to satisfy at six different stages simultaneously, what RecursiveCharacterTextSplitter actually does under the hood, and what the 2024–2026 toolbox — late chunking, contextual retrieval, contextualized chunk embeddings, BGE-M3 multi-functionality, ColBERT late interaction — is actually for.

chunking
rag

Read

Updated May 26, 2026

Shorts

HNSW: How Vector Search Stays Logarithmic

The graph that turned vector search from O(N) into O(log N). A hierarchy of small-world graphs, three tunable parameters, and the recall/latency trade-off underneath every production vector database.

embeddings
hnsw
ann
vector-databases

Read

Jun 7, 2026

Shorts

BM25 vs Dense Embeddings: Why Hybrid Retrieval Wins

Two retrieval philosophies — lexical evidence vs learned geometry — fail in exactly opposite ways. That's why production retrieval is rarely pure dense, and why Reciprocal Rank Fusion ended up everywhere.

embeddings
retrieval
bm25
hybrid

Read

Jun 7, 2026

Shorts

Bi-encoder vs Cross-encoder: Why Retrieval Is Two-Stage

A bi-encoder encodes query and document independently; a cross-encoder reads them together. One is cheap and slightly dumb, the other is expensive and much sharper. Modern retrieval uses both — and the two-stage cascade is the whole point.

embeddings
retrieval
reranking

Read

Jun 7, 2026

Shorts

Cosine, Dot Product, Euclidean: When Are They the Same Thing?

Three different similarity measures that, on normalized embeddings, all rank the same. Why cosine is the convention, why dot is the implementation, and when the choice actually matters.

embeddings
similarity
retrieval

Read

May 28, 2026