Aman Madaan personal blog
Notes on Weight Initialization for Deep Neural Networks
Neural networks involve long sequence of multiplications, usually between a matrix and a vector, say \(a*x\). The result of this sequence of multiplications will either have a huge magnitude or be reduced to 0. We can divide \(a\) by a number ( scaling_factor ) to scale down its magnitude to the right level. Proper…
- Points
- 117
- HN comments
- 18
- Submissions
- 1
- Cited by
- 0 commenters
AI & Machine LearningMath & Algorithms2019
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| May 20, 2019 | Notes on Weight Initialization for Deep Neural Networks by axiom92 | 117 | 18 |
