gau-nernst's blog personal blog
Writing Speed-of-Light Flash Attention for 5090 in CUDA C++
In this post, I will walk through how I learned to implement Flash Attention for 5090 in CUDA C++. The main objective is to learn writing attention in CUDA C++, since many features are not available in Triton, such as MXFP8 / NVFP4 MMA for sm120. I also feel this is a natural next step after learning about matmul…
- Points
- 159
- HN comments
- 34
- Submissions
- 1
- Cited by
- 0 commenters
AI & Machine LearningSoftware Engineering2025
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| Aug 23, 2025 | Writing Speed-of-Light Flash Attention for 5090 in CUDA C++ by dsr12 | 159 | 34 |
