Zhihao Jia

Compiling LLMs into a MegaKernel: A path to low-latency inference

TL;DR: We developed a compiler that automatically transforms LLM inference into a single megakernel — a fused GPU kernel that performs…