New

The latest blog posts to take off on Hacker News. 2,391 posts.

Math & Algorithms
Any year
163

The Q, K, V Matrices

At the core of the attention mechanism in LLMs are three matrices: Query, Key, and Value. These matrices are how transformers actually pay attention to different parts of the input. In this write-up, we will go through the construction of these matrices from the ground up.

166

Parsing Advances

I find myself writing yet another toy parser, as one does during a Christmas break. It roughly follows Resilient LL Parsing Tutorial. Not because I need resilience, but mostly because I find producing a syntax tree and a collection of diagnostics a more natural fit for the problem than bailing out on the first error.

173

50 years of proof assistants

Crackpots ranging from billionaire Peter Thiel to random YouTube influencers claim that science has been stagnating for the past 50 years. They admit that computing is an exception: they don’t pretend that my personal 32GB laptop is not an advance over the 16MB mainframe that served the whole Caltech community when I…

175

Beautiful Abelian Sandpiles

I can't remember where I first saw them, but ever since, I have been unable to forget them: abelian sandpiles. I'm far from the only one . They're remarkably simple, yet produce lovely symmetric patterns. I loved them so much that I adorned the title banner of this blog with an animation of an abelian sandpile. But…

176

My favourite small hash table

I'm the kind of person who thinks about the design and implementation of hash tables. One design which I find particularly cute, and I think deserves a bit more publicity, is Robin Hood open-addressing with linear probing and power-of-two table size. If you're not familiar with hash table terminology, that might look…