Sebastian Ruder personal blog
An overview of gradient descent optimization algorithms (2016)
Gradient descent is the preferred way to optimize neural networks and many other machine learning algorithms but is often used as a black box. This post explores how many of the most popular gradient-based optimization algorithms such as Momentum, Adagrad, and Adam actually work.
- Points
- 135
- HN comments
- 29
- Submissions
- 2
- Cited by
- 0 commenters
AI & Machine LearningMath & Algorithms2025
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| Jan 23, 2025 | An overview of gradient descent optimization algorithms (2016) by skidrow | 135 | 26 |
| Jun 10, 2018 | An overview of gradient descent optimization algorithms by bjourne | 62 | 3 |
