lenatriestounderstand
Start learning

Section 7 of 9

Embeddings and Retrieval

Embeddings, similarity and nearest-neighbor search, ANN indexes and vector databases, sparse and hybrid retrieval, chunking, reranking, and the search lineage from BoW to modern embedding models.

9 items in this section.

Long reads

01
Lab inside

Embeddings: How Geometry Pretends to Be Meaning

Embeddings aren't an encoding of text — they're an attempt to make geometry behave as if it carried meaning. What it means to compress text into a fixed-length vector, how contrastive learning turns statistical structure into distances and directions, why cosine similarity works (and when it stops), how dimension, chunking, context window, and reranking change the physics of a retrieval pipeline, and where embeddings lie usefully.

  • embeddings
Read
Updated Jun 8, 2026
02

How Text Became Geometry

Sixty years of incremental work behind the modern embedding vector. From bag-of-words and BM25, through Word2Vec's distributional hypothesis, to contextual embeddings and contrastive retrieval models. The point isn't trivia — it's that each older idea is still in production today, and the picture only makes sense once you've seen the road that led here.

  • embeddings
  • history
  • retrieval
Read
Updated May 21, 2026
03
Lab inside

Chunking Strategies

Chunking turns out to be more architectural than it looks. A walk through what a chunk has to satisfy at six different stages simultaneously, what RecursiveCharacterTextSplitter actually does under the hood, and what the 2024–2026 toolbox — late chunking, contextual retrieval, contextualized chunk embeddings, BGE-M3 multi-functionality, ColBERT late interaction — is actually for.

  • chunking
  • rag
Read
Updated May 26, 2026

Shorts

Shorts

Chunking Strategies, on One Page

RAG's quietest failure mode isn't the model or the embedding — it's where you cut the text. Chunk too big and retrieval goes fuzzy; too small and you sever an answer from the condition that qualifies it.

  • rag
  • chunking
  • retrieval
  • embeddings
  • +1
Read
Oct 27, 2026
Shorts

In-Batch Negatives: Where a Retrieval Model Gets Its Contrast

A contrastive model learns from pairs it is given and from the far larger number of pairs it is not: every other example in the batch becomes a negative. That trick is why batch size is a quality knob, and why the temperature next to it is doing real work.

  • embeddings
  • retrieval
  • contrastive-learning
  • training
Read
Oct 27, 2026
Shorts

HNSW: How Vector Search Stays Logarithmic

The graph that turned vector search from O(N) into O(log N). A hierarchy of small-world graphs, three tunable parameters, and the recall/latency trade-off underneath every production vector database.

  • embeddings
  • hnsw
  • ann
  • vector-databases
Read
Jun 7, 2026
Shorts

BM25 vs Dense Embeddings: Why Hybrid Retrieval Wins

Two retrieval philosophies — lexical evidence vs learned geometry — fail in exactly opposite ways. That's why production retrieval is rarely pure dense, and why Reciprocal Rank Fusion ended up everywhere.

  • embeddings
  • retrieval
  • bm25
  • hybrid
Read
Jun 7, 2026
Shorts

Bi-encoder vs Cross-encoder: Why Retrieval Is Two-Stage

A bi-encoder encodes query and document independently; a cross-encoder reads them together. One is cheap and slightly dumb, the other is expensive and much sharper. Modern retrieval uses both — and the two-stage cascade is the whole point.

  • embeddings
  • retrieval
  • reranking
Read
Jun 7, 2026
Shorts

Cosine, Dot Product, Euclidean: When Are They the Same Thing?

Three different similarity measures that, on normalized embeddings, all rank the same. Why cosine is the convention, why dot is the implementation, and when the choice actually matters.

  • embeddings
  • similarity
  • retrieval
Read
May 28, 2026