Lena Tries
To Understand

Isotonic Regression on GPU

torchisotonic and the paper behind it, in plain words. A curve that may only go up is a ten-line walk on a CPU, and a walk is the one thing a GPU cannot share out: on ten million numbers a far bigger GPU can be even slower. The way out is to stop walking and draw instead, since the fit is a string pulled tight under the running totals, and then to stop copying the pieces of that string that never change.

• gpu
• cuda
• isotonic-regression
• calibration
• +4

PagedAttention: Treating KV Memory Like Virtual Memory

A conversation's key/value cache grows one token at a time and nobody knows how long it will get. Give every request a contiguous buffer sized for the worst case and one Tesla T4 holds 573 of them, most of that memory reserved for answers that never arrive. Hand the same card out in blocks of 16 tokens and it holds several thousand. Measured on vLLM 0.29: what the block table costs the attention kernel, how the engine decides that two requests hold the same block, how much of the card the cache actually gets, and how many sequences all of it makes room for.

• llm
• inference
• kv-cache
• memory
• +4

Where Your Prompt Goes: Inside a Modern AI GPU

Follow one message into a 2026 AI server: through CPU memory, HBM4, compute dies, matrix units and the links between GPUs, then back out as an answer. Original interactive diagrams show where weights already live, what a request creates, and what the hardware waits for. Based on NVIDIA Rubin and AMD CDNA 5.

• gpu
• rubin
• vera
• cdna-5
• +4

Knowledge Distillation: What a Teacher Actually Teaches

Training a small model on a large model's probabilities instead of the true labels often gives a better small model. Why that works, taken apart one ingredient at a time; what happens when the labels the teacher learned from were wrong; whether a stronger teacher makes a stronger student; what to copy, on which data and about which view of it; how long to train and how big the student should be before any of this matters; why copying the teacher and being right are different numbers; how a student can beat its teacher and which of its flaws it inherits; what changes for language models without logits; and when distillation pays for itself — measured on CIFAR-10 and on a small language model trained on this site's notes.

• knowledge-distillation
• model-compression
• kl-divergence
• regularization
• +3

Chunking Strategies, on One Page

RAG's quietest failure mode isn't the model or the embedding — it's where you cut the text. Chunk too big and retrieval goes fuzzy; too small and you sever an answer from the condition that qualifies it.

• rag
• chunking
• retrieval
• embeddings
• +1

Arithmetic Intensity: Why Most Kernels Never Touch the Peak

A GPU's advertised TFLOPs describe one of two limits. Which one an operation actually hits is decided by a single ratio — FLOPs per byte moved — and most operations in a neural network sit far below the crossover.

• gpu
• roofline
• memory-bandwidth
• arithmetic-intensity
• +1

When a CUDA Call Returns — and When the Work Is Actually Done

Most GPU calls return before the GPU has done anything. That is the point — and it is also why a buffer can hold the previous batch's results, why a free can stall for 50 ms, and why an 'async' copy sometimes is not.

• cuda
• gpu
• async
• streams
• +1

LoRA: Fine-Tuning as a Low-Rank Delta

LoRA does not train a model. It trains a small matrix that is added to a frozen one — and the rank of that matrix is a hard limit on how many independent directions the fine-tune is allowed to move in.

• llm
• fine-tuning
• lora
• peft

Isotonic Regression on GPU

torchisotonic and the paper behind it, in plain words. A curve that may only go up is a ten-line walk on a CPU, and a walk is the one thing a GPU cannot share out: on ten million numbers a far bigger GPU can be even slower. The way out is to stop walking and draw instead, since the fit is a string pulled tight under the running totals, and then to stop copying the pieces of that string that never change.

From Python Call to GPU Kernel: The C++ Underneath ML Libraries

One line of Python — index.search(query, k=10) — taken apart layer by layer, with a small nearest-neighbour library written in C++ and CUDA to take it apart with. Where numbers live and who frees them, what a copy of a gigabyte costs, why the order of additions changes both the speed and the answer, when an abstraction is free and when it is ruinous, what a template costs to compile, what happens between the call returning and the GPU finishing — and why the same search runs 25 times faster after eight rewrites that change nothing about what it computes. Measured end to end on a Kaggle T4.

RLCR from Scratch: A One-Pass Decision Model with Honest Probabilities

A model reads a support ticket, picks a department, and returns 0.94. RLCR — training that number with a reward only an honest probability can maximise — and a small typed-decision model built from an ordinary encoder: one forward pass, every option a parallel branch, and the same weights trained seven ways to find out what the reinforcement learning actually adds.

How LLM Inference Actually Works: Where the Time Goes on a Real GPU

What a GPU is actually doing while a language model answers: the generation loop, the prefill pass that reads the prompt and the decode steps that write the answer, the two waits a user feels — time to first token and time per output token — and why the second is limited by moving the model's weights through memory. Measured end to end on a Tesla T4, including why a token costs nine times what its bytes say, what a real kernel trace shows instead, and how latency and throughput pull apart when many people share one card.