1
Fast LLM Inference From Scratch (using CUDA)
This post is about building an LLM inference engine using C++ and CUDA from scratch without libraries.
Andrew Chan
I'm a software engineer living in San Francisco. Previously I was at South Park Commons and Figma , and graduated from UC Berkeley in 2019. This is my blog where I post about things I'm interested in, mostly tech-related.
- Popular posts
- 4
- Total points
- 1,642
- HN comments
- 315
- Rank
- #1,092 personal
Visit andrewkchan.dev → On Hacker News Dec 2023 – Feb 2026
Systems & Performance 2AI & Machine Learning 2Software Engineering 1Math & Algorithms 1Infrastructure & DevOps 1Games & Graphics 1
2
Diffusion Models
I spent 2022 learning to draw and was blindsided by the rise of AI art There is lots to say about this, whether it has been or will be a good thing for artists and society in the long run. I hope to write about it in another post. models like Stable Diffusion. Suddenly, the computer was a better artist than I could…