1
A Mechanistic Interpretability Analysis of Grokking
Neel Nanda reverse engineers neural networks that have "grokked" modular addition, showing that they operate using Discrete Fourier Transforms and tr…
A community blog devoted to technical AI alignment research
Visit alignmentforum.org → RSS feedOn Hacker News Dec 2019 – Nov 2023
Math & Algorithms 4AI & Machine Learning 3Life, Learning & Writing 1
Neel Nanda reverse engineers neural networks that have "grokked" modular addition, showing that they operate using Discrete Fourier Transforms and tr…
Double descent is a puzzling phenomenon in machine learning where increasing model size/training time/data can initially hurt performance, but then i…