Mitch Gordon personal blog
RETRO is fast
When I first read Google’s RETRO paper, I was skeptical. Sure, RETRO models are 25x smaller than the competition, supposedly leading to HUGE savings in training and inference costs. But what about the new trillion token “retrieval database” they added to the architcture? Surely that must add back some computational…
- Points
- 141
- HN comments
- 20
- Submissions
- 1
- Cited by
- 0 commenters
AI & Machine LearningSystems & Performance2022
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| Jul 3, 2022 | RETRO is fast by bilsbie | 141 | 20 |
