Amazon Science company blog
Do large language models need all those layers?
Finding that 70% of attention heads and 20% of feed-forward networks can be excised with minimal effect on in-context learning suggests that large language models are undertrained.
- Points
- 178
- HN comments
- 82
- Submissions
- 1
- Cited by
- 0 commenters
AI & Machine LearningSoftware Engineering2023
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| Dec 15, 2023 | Do large language models need all those layers? by belter | 178 | 82 |
