Lilian Weng personal blog
Large Transformer Model Inference Optimization
[Updated on 2023-01-24: add a small section on Distillation.] Large transformer models are mainstream nowadays, creating SoTA results for a variety of tasks. They are powerful but very expensive to train and use. The extremely high inference cost, in both time and memory, is a big bottleneck for adopting a powerful…
- Points
- 136
- HN comments
- 20
- Submissions
- 1
- Cited by
- 0 commenters
AI & Machine LearningSystems & Performance2023
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| Jan 20, 2023 | Large Transformer Model Inference Optimization by headalgorithm | 136 | 20 |
