Blog
- Score Centering: Stabilize RL TrainingHow Thinking Machines' hidden StableReinforce loss was reverse-engineered from its gradients, and a first-principles derivation of the method it turned out to be: score centering, from Marek & Ryabinin (2026).
- What Actually Changes When a Network GroksGrokking on a new non-abelian group task, and an anatomy of the transition: the collapse of a memorising lookup table onto about three irreducible representations, gated by weight decay, timed by the learning rate, and preceded by no warning at all.
- What RMSNorm Actually Does (And Why You Can't Replace It With a Point-wise Norm)An empirical dissection of normalization in transformers, and why the backward coupling, not the forward rescaling, is what makes RMSNorm work.