Top posts

Every blog post that took off on Hacker News, ranked by the votes of its best submission. 3,091 posts.

AI & Machine Learning
Any year
1339

The Q, K, V Matrices

At the core of the attention mechanism in LLMs are three matrices: Query, Key, and Value. These matrices are how transformers actually pay attention to different parts of the input. In this write-up, we will go through the construction of these matrices from the ground up.

1349

Money bubble

I think I’m probably going to lose quite a lot of money in the next year or two. It’s partly AI’s fault, but not mostly. Nonetheless I’m mostly going to write about AI, because it intersects the technosphere, where I’ve lived for decades.