Topics /

Hardware & Electronics

The most popular blog posts about hardware & electronics on Hacker News. 474 posts.

2023
77

Making AMD GPUs competitive for LLM inference

MLC-LLM makes it possible to compile LLMs and deploy them on AMD GPUs using ROCm with competitive performance. More specifically, AMD Radeon™ RX 7900 XTX gives 80% of the speed of NVIDIA® GeForce RTX™ 4090 and 94% of the speed of NVIDIA® GeForce RTX™ 3090Ti for Llama2-7B/13B. Besides ROCm, our Vulkan support allows us…