Lilian Weng personal blog
How to train large models on many GPUs? (2021)
[Updated on 2022-03-13: add expert choice routing.] [Updated on 2022-06-10]: Greg and I wrote a shorted and upgraded version of this post, published on OpenAI Blog: “Techniques for Training Large Neural Networks”
- Points
- 216
- HN comments
- 33
- Submissions
- 1
- Cited by
- 0 commenters
AI & Machine LearningSystems & Performance2023
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| Feb 11, 2023 | How to train large models on many GPUs? (2021) by eternalban | 216 | 33 |
