New

The latest blog posts to take off on Hacker News. 80,815 posts.

Any topic
Any year
5810

A4 Paper Stories

I sometimes resort to a rather common measuring technique that is neither fast, nor accurate, nor recommended by any standards body and yet it hasn't failed me whenever I have had to use it. I will describe it here, though calling it a technique might be overselling it. Please do not use it for installing kitchen…

5815

The Q, K, V Matrices

At the core of the attention mechanism in LLMs are three matrices: Query, Key, and Value. These matrices are how transformers actually pay attention to different parts of the input. In this write-up, we will go through the construction of these matrices from the ground up.