1
Fast LLM Inference From Scratch (using CUDA)
This post is about building an LLM inference engine using C++ and CUDA from scratch without libraries.
I'm a software engineer living in San Francisco. Previously I was at South Park Commons and Figma , and graduated from UC Berkeley in 2019. This is my blog where I post about things I'm interested in, mostly tech-related.
Visit andrewkchan.dev → On Hacker News Dec 2023 – Feb 2026
Systems & Performance 2AI & Machine Learning 2Software Engineering 1Math & Algorithms 1Infrastructure & DevOps 1Games & Graphics 1
This post is about building an LLM inference engine using C++ and CUDA from scratch without libraries.
For some reason, nobody's written about what it takes to crawl a big chunk of the web in a while: the last point of reference I saw was Michael Nielsen's post from 2012 .