SeriesComplete6 of 6 articles
Parameter Golf: Learning Transformers Under 16MB
Learning transformer design under a 16MB limit through compression, evaluation, and agent-assisted experiments.
- 01The 16MB BetI studied Parameter Golf because I wanted a real constraint to force me past surface-level transformer knowledge.
- 02Learning The Parameter Golf Game BoardThe first useful lesson was that Parameter Golf is not scored on vibes. It is scored on a very specific artefact pipeline.
- 03The Bottleneck Kept MovingBetter model quality did not always produce a better submission. Compression and evaluation kept changing what progress meant.
- 04What Tiny Transformers PunishA small transformer under a byte budget makes comfortable architecture defaults feel expensive.
- 05Agents As Lab InfrastructureCoding agents were most useful when I stopped treating them as geniuses and started treating them as structured lab infrastructure.
- 06The Next Parameter Golf RunIf I ran Parameter Golf again, I would spend less time chasing cleverness and more time making the bottleneck visible earlier.