1
NanoGPT Slowrun: Language Modeling with Limited Data, Infinite Compute
NanoGPT Slowrun is an open effort to implement data-efficient learning algorithms; 5.5x data efficiency in the first week and improving.
Q Labs
Q is a research lab working on moonshots in deep learning: new learning algorithms that scale to unimaginable amounts of compute, even with little data.
- Popular posts
- 3
- Total points
- 633
- HN comments
- 163
- Rank
- #2,156 company
Visit qlabs.sh → On Hacker News Mar 2026 – Oct 2026
AI & Machine Learning 3Databases & Data 2Math & Algorithms 1
2
NanoGPT Slowrun: 10x Data Efficiency with Infinite Compute
We've achieved 10x data efficiency with NanoGPT Slowrun within a few weeks. An ensemble of 1.8B parameter models (18B total params) trained on 100M tokens matches what would normally require 1B tokens with a standard LM baseline. Data efficiency matters because compute grows much faster than data . Since our current…