Shorts
Single-idea notes drawn from the longer pieces. 1-2 minutes each. Use the search to narrow by topic, tag, or track — or browse the full feed →.
57 shorts
RAG's quietest failure mode isn't the model or the embedding — it's where you cut the text. Chunk too big and retrieval goes fuzzy; too small and you sever an answer from the condition that qualifies it.
A GPU's advertised TFLOPs describe one of two limits. Which one an operation actually hits is decided by a single ratio — FLOPs per byte moved — and most operations in a neural network sit far below the crossover.
Most GPU calls return before the GPU has done anything. That is the point — and it is also why a buffer can hold the previous batch's results, why a free can stall for 50 ms, and why an 'async' copy sometimes is not.
LoRA does not train a model. It trains a small matrix that is added to a frozen one — and the rank of that matrix is a hard limit on how many independent directions the fine-tune is allowed to move in.
One runs a live loop with a reward model and generation inside training; the other trains on a file of pairs. They are not different goals — they are different placements of the same objective, and the difference shows up operationally before it shows up in quality.
The two layers differ in one axis — statistics across the batch, or across the features of one example — and every practical difference between them, including why transformers use one and not the other, follows from that.
Networks represent more concepts than they have dimensions by storing them as almost-orthogonal directions and tolerating the interference. That is why single neurons look like nonsense, and why sparse autoencoders exist.
A contrastive model learns from pairs it is given and from the far larger number of pairs it is not: every other example in the batch becomes a negative. That trick is why batch size is a quality knob, and why the temperature next to it is doing real work.
Silhouette rewards compact, well-separated blobs. When the real structure is not blob-shaped, the clustering that scores highest can be the one that is most wrong — and there are no labels to tell you.
At temperature 0 sampling becomes argmax, so the same prompt should give the same answer. In production it often does not — and the reasons are in the hardware and in the batch you are sharing, not in the model.
Time series is one entity over time. Panel data is many entities over time. The new dimension — unit-level heterogeneity — is what panel methods exist to handle.
Random k-fold CV reshuffles time. On time series that means training on the future to predict the past — and quietly inflating every metric. The correct alternatives are expanding-window and rolling-window CV.
Three components, one additive identity. STL splits a series into trend, seasonality, and residual so you can see what's actually in it before modeling — and the same decomposition idea propagates into modern deep architectures like N-BEATS.
Everyone trains classifiers with cross-entropy, far fewer can say why that loss and not another. It isn't a design choice — it falls out of one principle: maximize the probability the model assigns to the data. Cross-entropy, log-loss, MSE, and KL are the same idea wearing four different hats.
KL divergence turns up in cross-entropy training, VAEs and variational inference, knowledge distillation, and the RLHF leash. That isn't a coincidence — a huge fraction of ML is secretly the sentence 'make distribution q close to distribution p,' and KL is what that sentence reduces to.
Run a regression on anything and it hands back a number. That number is the causal effect only if the regressor and the error term don't move together. Endogeneity is the name for that assumption breaking — and why more data can't fix it.
Two words that sound like heavy jargon but name one yes/no question: does your regressor move with the error term? Clean is exogenous, contaminated is endogenous — and 'strict' just extends the question across time.
If your regressor is contaminated, borrow a clean source of variation that pushes it around. That borrowed lever is an instrument, and two-stage least squares is the machine that uses it.
Watch the same units over time and you get a free superpower: you can subtract away everything about a unit that never changes — including the confounders you can't even measure.
A cutoff nobody chose for your benefit — a passing score, an income line, an age limit — is a natural experiment hiding in plain sight. Just above and just below the line, units are near-identical except for the treatment.
Two-stage least squares replaces the dirty regressor with a clean copy. The control function does the opposite — it keeps the dirty regressor and adds one column that soaks up the dirt. In the linear case they land on the exact same number.
Elasticity is the percent change in quantity per percent change in price. It looks like one tidy regression of quantity on price — and that regression is wrong in the exact direction that costs you money.
The treatment lifts conversion 3%. For whom? An average is one number, and a single number can't tell 'helps everyone a little' apart from 'helps half a lot and quietly harms the rest.'
Some of the future is already known — holidays, a scheduled promo, next week's price. Exogenous (known-future) variables are how you tell a forecaster about it, and every model family, from gradient boosting to Prophet to deep forecasters, has its own channel for them.
Distillation temperature softens the teacher and the student at once — a different operation from the sampling temperature in text generation. The T² in front of the loss is not a tuning constant: it cancels a gradient that would otherwise shrink as the square of the temperature.
In distillation the student learns the teacher's answers at the points it is asked about, and nothing else. That set of points needs no labels, is a design decision rather than raw material, and moved the result more than the choice of teacher did.
A distilled model can be graded on how closely it reproduces its teacher, or on how often it is right. With a strong teacher the two look like the same number. They come apart exactly where it matters, and each answers a question the other cannot.
Training a small model on a big model's text and training it on the big model's next-token distributions are not the same procedure. One is a noisy Monte Carlo version of the other — and training on the teacher's single most likely continuation is neither.
Distil a network into a copy of itself, same architecture, same data, no compression anywhere — and it can come out better than the model it learned from. Whether it does depends on how much the original memorized.
Reading the prompt and writing the answer are the same weights running twice, but they are different workloads for the hardware. The textbook split is compute-bound against memory-bound — and at batch 1 the measurement says neither.
A trained classifier answers with one class and a list of numbers nobody reads. Those numbers say which wrong answers it considered nearly right, no label can carry them, and they are most of what a distilled student learns from.
A matrix turns and stretches every arrow in space — except for a few special directions it only stretches. Those are eigenvectors, the natural grain of the transformation.
Once you know a matrix's un-turnable directions, you can rewrite the whole thing as three simple moves: rotate, stretch, rotate back. That's eigendecomposition.
Eigenvectors are beautiful on paper. Computing them for a 100,000×100,000 matrix is an engineering problem — and the tool that does it, iteratively, is an eigensolver.
Supervised fine-tuning is the imitation step of post-training: show the model (prompt → ideal answer) pairs and minimize cross-entropy on the target tokens. It teaches the format of being an assistant — and hits a ceiling that preference learning exists to break.
Reinforcement Learning from Human Feedback is how a model is bent from optimizing likelihood to optimizing preference: an SFT base, a reward model trained on human comparisons, and a policy optimized against that reward under a KL leash.
Reinforcement Learning from AI Feedback swaps the human labeler for a model: a capable model judges which of two responses is better, and those AI preferences train the reward. Constitutional AI is its most influential form — and the values don't disappear, they move and become explicit.
GQA sits between full multi-head attention and MQA: query heads are partitioned into a small number of groups, and each group shares one K/V set. Most of the KV-cache savings of MQA, most of the head diversity of MHA — and a cheap conversion path from existing checkpoints.
Why long context costs memory, not flops. The KV-cache size formula has six factors — once you can read it, every attention-variant design choice falls into place.
The most famous LLM bug — miscounting letters in a word — is not a reasoning failure. It is a representation failure. The model isn't fed letters, it's fed tokens. Once you know what a token actually is, the bug is the only outcome that could have happened.
Three sampling knobs, three different operations on the same next-token distribution. Temperature reshapes; top_k truncates by count; top_p truncates by cumulative probability. They are not interchangeable.
The graph that turned vector search from O(N) into O(log N). A hierarchy of small-world graphs, three tunable parameters, and the recall/latency trade-off underneath every production vector database.
Two retrieval philosophies — lexical evidence vs learned geometry — fail in exactly opposite ways. That's why production retrieval is rarely pure dense, and why Reciprocal Rank Fusion ended up everywhere.
A bi-encoder encodes query and document independently; a cross-encoder reads them together. One is cheap and slightly dumb, the other is expensive and much sharper. Modern retrieval uses both — and the two-stage cascade is the whole point.
The same data, two layouts on disk. For analytical queries that touch a few columns out of many, Parquet reads tens of times less from disk than CSV — and that gap is structural, not a benchmark artifact.
Partition count is the ceiling on parallel consumption. Ordering is a per-partition guarantee, not a per-topic one. Two facts that decide most Kafka architecture questions.
Everyone knows attention is softmax(QKᵀ/√d)V. Fewer notice it's structurally the same thing as kernel regression — Nadaraya-Watson smoothing from 1964, with a learned similarity. Seeing the kernel underneath tells you immediately how to make attention linear.
The evidence lower bound looks like a formula handed down from on high. It isn't invented — it's forced. When the evidence integral is intractable but you still want maximum likelihood, there is essentially one move available, and the ELBO is what it produces.
Three different similarity measures that, on normalized embeddings, all rank the same. Why cosine is the convention, why dot is the implementation, and when the choice actually matters.
TCN stands for Temporal Convolutional Network — a causal CNN with dilated convolutions that gives sequence models exponential receptive field for the cost of linear depth.
Univariate is one signal over time. Multivariate is many signals over time. The decision between them is about whether the extra signals actually carry information about the target.
ARIMA in three letters: AutoRegression, Integration, Moving Average. SARIMA adds seasonal versions of each. Together they handle the linear, stationary case — which covers a surprising amount of forecasting.
Why production LLM decoders share K and V across attention heads — and the 8–32× memory cut that follows.
An LLM agent is an LLM wrapped in a loop that gives it tools, state, and a stopping rule. The architecture is the shape of that loop.
MLA stores a low-rank latent representation for each token and uses learned projections to produce the K/V information needed by attention at decode time. A different way to attack the KV-cache problem than MQA or GQA — trading a sharing bottleneck for a rank one.
Each interpolation method makes a different assumption about what was happening between known measurements. The right method depends on what the signal actually does between samples, not on what's the default.
TS preprocessing is a pipeline, not a checklist. The order of steps changes the result — scaling before handling missing values is the common trap.