Evan Miller personal blog
Attention Is Off By One
Let’s fix these pesky Transformer outliers using Softmax One and QuietAttention.
- Points
- 1,040
- HN comments
- 331
- Submissions
- 1
- Cited by
- 3 commenters
AI & Machine LearningSoftware Engineering2023
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| Jul 24, 2023 | Attention Is Off By One by elbasti | 1,040 | 331 |
Cited in comments
Linked from 4 Hacker News comments by 3 different people, from Feb 2024 to Feb 2026.
Most recent:
- wizzwizz4 · Feb 28, 2026
- canjobear · Aug 8, 2025
- wizzwizz4 · Jun 20, 2024
- TrueDuality · Feb 13, 2024
