Latest
The full feed, sorted by the most recent update. Use the filter to narrow by topic, tag, or track — or browse by category →.
34 notes
Endogeneity — what it is formally, why OLS becomes biased and inconsistent under it, the three main sources (omitted variables, simultaneity, measurement error) plus self-selection, how to detect it in practice, and why this is the central problem the rest of the Econometrics track is built around.
Instrumental Variables / 2SLS as the foundational tool for causal inference in observational data when the treatment is endogenous — the IV idea in plain terms, the four conditions a valid instrument must satisfy (relevance, exclusion, independence, monotonicity), the two-stage estimator, weak-instrument diagnostics beyond the F>10 rule, what IV actually identifies (LATE, not ATE), and worked instrument choices for pricing problems.
Regression discontinuity as a quasi-experimental method built on a threshold-based assignment rule — the continuity assumption that does the identification work, sharp and fuzzy variants, the modern bandwidth and diagnostic toolkit, and why the resulting effect is local to the cutoff rather than population-wide.
The control function approach as the residual-based alternative to 2SLS — when the two agree, what CF's structural assumption actually is, where an ML-flexible first stage helps and where it does not, and how the idea extends through cross-fitting into double / debiased machine learning.
Fixed effects for panel data — the within-transformation, what it controls for and what it misses, cluster-robust standard errors, the LSDV equivalence, the staggered-DiD problem, and when FE is the right tool.
Causal ML at the meeting point of econometrics and ML — ATE vs CATE, uplift modelling, DML and orthogonal-moments inference, causal forests, counterfactual prediction, off-policy evaluation, and the standard mistakes from treating predictive models as causal.
Pricing as the worked-example for the Econometrics track — why price is endogenous, the identification strategies (IV, FE, RD, DML, CATE), cross-price elasticity and cannibalization, and what-if analysis with its constant-elasticity caveats.
Fine-tuning is not prompt repair. It is a decision to write a reusable parameter delta into an existing checkpoint. That delta changes future logits, defaults, and trade-offs. This note is about when that is worth doing: what fine-tuning actually buys, how to tell whether a gap belongs in the weights, and why data, evals, forgetting, and probability shape matter more than the slogan 'just fine-tune it'.
Post-training is not one fine-tuning method. It is a sequence of objective signals. Continued pretraining teaches substrate, SFT teaches examples and defaults, preference optimization teaches comparisons, RLVR teaches verifiable trajectories, and distillation transfers the resulting behavior. The important design question is not which acronym is fashionable, but which stage matches the behavior you are trying to install.
A reference-style deep dive into the modern knobs around post-training: preference optimization variants, LoRA and PEFT methods, memory-efficient full fine-tuning, model merging, distillation, tooling, and serving. Read this after the pipeline note, once you know which stage you actually need.
What happens inside a transformer when you send a prompt, and how the practical knobs — temperature, max_tokens, structured outputs, batching strategy, retry-with-catch-up — fall out of that picture.
Agent architecture is where LLM engineering stops being mostly about prompts and starts looking like distributed systems. Covers workflows vs agents, the classical loop, five paradigms (ReAct, Function Calling, Plan-and-Execute, Reflection, CodeAct), MCP as the protocol layer above per-vendor function calling, multi-agent patterns, computer use, memory and resumability, production failure modes including indirect prompt injection, tool security, cost levers, and observability.
Chunking turns out to be more architectural than it looks. A walk through what a chunk has to satisfy at six different stages simultaneously, what RecursiveCharacterTextSplitter actually does under the hood, and what the 2024–2026 toolbox — late chunking, contextual retrieval, contextualized chunk embeddings, BGE-M3 multi-functionality, ColBERT late interaction — is actually for.
Sixty years of incremental work behind the modern embedding vector. From bag-of-words and BM25, through Word2Vec's distributional hypothesis, to contextual embeddings and contrastive retrieval models. The point isn't trivia — it's that each older idea is still in production today, and the picture only makes sense once you've seen the road that led here.
Embeddings aren't an encoding of text — they're an attempt to make geometry behave as if it carried meaning. What it means to compress text into a fixed-length vector, how contrastive learning turns statistical structure into distances and directions, why cosine similarity works (and when it stops), how dimension, chunking, context window, and reranking change the physics of a retrieval pipeline, and where embeddings lie usefully.
A practical guide to the post-training lifecycle of deep learning forecasters, covering SHAP-based prediction explanations, lightweight recalibration for drift, and the broader production-maintenance roadmap.
Saying a model was 'trained on text written before T' invites a picture of human knowledge as of T. The actual corpus is volumetrically skewed toward recent years, dominated by retroactively-edited sources like Wikipedia, missing reliable per-document timestamps, and survivor-biased for older periods. The mechanisms, the failure modes that fall out, what's silently absent from datasheets, and what time-aware pretraining would have to do differently.
A chat transcript preserves order but not elapsed time, world state, or whether earlier hypotheses have expired. For long-running agents, temporal grounding is a runtime problem, not a model problem — what 'now' actually is, the failure modes that fall out when context gets treated as state, the primitives (clocks, event logs, state reducers, expectations, monitors) that close the gap, and how to measure whether it works.
Why modern LLMs are no longer just decoder-only transformers with standard multi-head attention. Attention has become a design space — MHA, MQA, GQA, MLA, sliding-window, sparse, linear, recurrent, hybrid — plus position encoding, attention sinks, and KV-cache compression. Each variant solves a different bottleneck.
What separates a working LLM prompt from a flaky one in 2026 — instruction hierarchy, in-context learning, chain-of-thought, structured outputs, reasoning-model specifics, and the prompt-injection trust boundary.
What hallucination looks like at the level of the transformer's internal computation — distributed representations, signal competition in the residual stream, the softmax bottleneck, the activation-output gap, and the architectural reasons there is no first-class epistemic channel.
A practical checklist for training deep learning time-series models, covering windowing, initialization, embeddings, regularization, optimizers, learning-rate schedules, loss functions, and the Keras training workflow.
A practical guide to ARIMA, SARIMA, and SARIMAX: how classical statistical forecasting models handle autocorrelation, differencing, seasonality, external regressors, diagnostics, and where they still fit in modern forecasting.
An overview of deep learning architectures for time-series forecasting: LSTM, TCN, DeepAR, N-BEATS, and TFT, with a focus on input shapes, local vs global training, covariates, probabilistic outputs, and practical model selection.
How recurrent networks model sequences: vanilla RNN, the vanishing-gradient problem, the LSTM gating mechanism, cell state, and the practical use of return_sequences in Keras.
A practical explanation of Temporal Convolutional Networks for time-series forecasting: causal convolutions, dilations, receptive field, residual blocks, optional GLU gating, output heads, and when TCN is useful as a standalone or hybrid forecasting component.
Playing with a hybrid forecasting architecture, assembled out of common neural building blocks: TCN front-end, LSTM recurrence, multi-head attention, TFT-style VSN feature routing with GRN, N-BEATS-style decomposition heads, seasonal and event branches.
Why CSV, JSON, and Parquet all coexist in real pipelines: file size, read speed, and tool compatibility rarely peak in the same format.
Kafka as a distributed log for streaming data transport, and Protobuf as a compact binary serialization format — how each works, when to use them, and why they pair well together.
Covers the object storage model — buckets, keys, versioning, lifecycle policies — and Minio as a self-hosted S3-compatible implementation.
A practitioner's overview of PostgreSQL: how it handles concurrency (MVCC), its type system, index types, and the Python tooling stack for working with it.
A practical guide to the main types of time series data and the common confusion between multivariate, panel, and exogenous-variable forecasting.
A hands-on explanation of how raw time series data is cleaned, regularized, interpolated, bucketed, and prepared for reliable modeling or visualization.
An introduction to statistical baseline methods for time-series anomaly detection: Z-Score, IQR, moving averages, simple ensembles, and evaluation metrics.