Model Mechanics

Raters preferred a 1.3-billion-parameter model over one with 175 billion. Alignment, RLHF, and why no answer comes out neutral.

Push a model past a perfect fit on its training data and test error starts falling again. Double descent, and what it did to learning theory.

A language model puts a probability on every possible next word, picks one, and runs the calculation again. Everything else follows from that loop.

Banks were scoring credit risk with models in the 1980s. A history of machine learning in finance, and why explainability came first.

Self-attention weighs every token at once, a convolution slides across pixels. Why one system writes a summary and another spots a tumor.