1
The maths you need to start understanding LLMs
A quick refresher on the maths behind LLMs: vectors, matrices, projections, embeddings, logits and softmax.
Giles Thomas
Currently mostly AI because in 2026 of course it is. Grinding through the nuts and bolts of LLMs and trying to work out why this stuff works...
- Popular posts
- 9
- Total points
- 3,212
- HN comments
- 1,014
- Rank
- #438 personal
Visit gilesthomas.com → RSS feedOn Hacker News Feb 2025 – Jun 2026
AI & Machine Learning 6Software Engineering 4Networking & Internet 2Life, Learning & Writing 2Infrastructure & DevOps 2Math & Algorithms 1
2
LLM from scratch, part 28 – training a base model from scratch on an RTX 3090
I felt like it should be possible to train a GPT-2 small level model on my own hardware using modern tools and open datasets from scratch. It was!
3
Writing an LLM from scratch, part 8 – trainable self-attention
Moving on from a toy self-attention mechanism, it's time to find out how to build a real trainable one. Following Sebastian Raschka's book 'Build a Large Language Model (from Scratch)'. Part 8/??
4
Writing an LLM from scratch, part 13 – attention heads are dumb
A pause to take stock: realising that attention heads are simpler than I thought explained why we do the calculations we do.
5
It’s still worth blogging in the age of AI
Although it might seem that AI will make it pointless, I still think it's worth blogging.
6
The benefits of learning in public
Learning in public helps me grow as an engineer and seems to benefit others too. Here's why I should do more.
7
Writing an LLM from scratch, part 22 – training our LLM
Finally, we train an LLM! The final part of Chapter 5 of Build an LLM (from Scratch) runs the model on real text, then loads OpenAI’s GPT-2 weights for comparison.
8
10Gb/s Ethernet: what I did to get it working in my home
I already had 2.5Gb/s working. Here's how I upgraded the whole house to 10Gb/s Ethernet -- including real iperf3 numbers, MikroTik switches, scary thermals, and what actually worked.
9
10Gb/s Ethernet: switching to a Broadcom SFP+ module
As predicted by several people, one of my Marvell-based SFP+ modules had overheating problems. I've switched it over to a Broadcom-based one, and it looks better, though I can't be certain.