Q Labs company blog
Dust: Pretraining Transformers Without Backpropagation
Deep learning has been built around backprop, the only credit assignment algorithm capable of training modern neural nets, including transformer-based language models. Backprop requires differentiability and produces first-order gradients, and deep learning’s architectures, optimizers, and hardware have co-evolved…
- Points
- 280
- HN comments
- 82
- Submissions
- 1
- Cited by
- 0 commenters
AI & Machine LearningMath & Algorithms2026
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| Oct 5, 2026 | Dust: Pretraining Transformers Without Backpropagation by E-Reverance | 280 | 82 |
