MLC AI Blog company blog
Making AMD GPUs competitive for LLM inference
MLC-LLM makes it possible to compile LLMs and deploy them on AMD GPUs using ROCm with competitive performance. More specifically, AMD Radeon™ RX 7900 XTX gives 80% of the speed of NVIDIA® GeForce RTX™ 4090 and 94% of the speed of NVIDIA® GeForce RTX™ 3090Ti for Llama2-7B/13B. Besides ROCm, our Vulkan support allows us…
- Points
- 354
- HN comments
- 345
- Submissions
- 2
- Cited by
- 0 commenters
AI & Machine LearningHardware & Electronics2023
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| Aug 9, 2023 | Making AMD GPUs competitive for LLM inference by djoldman | 354 | 132 |
| Dec 24, 2024 | Making AMD GPUs competitive for LLM inference (2023) by plasticchris | 313 | 213 |
