Topics /

AI & Machine Learning

The most popular blog posts about ai & machine learning on Hacker News. 762 posts.

2026
389

Dust: Pretraining Transformers Without Backpropagation

Deep learning has been built around backprop, the only credit assignment algorithm capable of training modern neural nets, including transformer-based language models. Backprop requires differentiability and produces first-order gradients, and deep learning’s architectures, optimizers, and hardware have co-evolved…