Visualizing Attention, a Transformer's Heart [video]
Demystifying attention, the key mechanism inside transformers and LLMs.
The latest blog posts to take off on Hacker News. 3,091 posts.
Demystifying attention, the key mechanism inside transformers and LLMs.
[custom_views_and_hn_comments hn_id=”40015185″] My startup Truss (gettruss.io) released a few LLM-heavy features in the last six months, and the narrative around LLMs that I read on Hacker News is now starting to diverge from my reality, so I thought I’d share some of the more “surprising” lessons after churning…
I’m a bit behind on my weeknotes, so there’s a lot to cover here. But first... a review of the last 24 hours of Large Language Model news. All times …
I had a really enlightening talk with Daniel Slutsky this week (who is an exceptional data scientist, software engineer, and community organizer I highly…
My comments on the NTIA AI Open Model Weights RFC
TL;DR Yes, albeit with patience.
Last Updated: January 23, 2026There’s no question that AI will impact jobs. But which jobs are more likely…
I attended the Story Discovery At Scale data journalism conference at Stanford this week. One of the perennial hot topics at any journalism conference concerns data extraction: how can we …
Looking in more detail at Google's first Tensor Processing Unit
Talk is cheap, show me the code Introduction A common problem my friends and I have is receiving actionable emails whose priorities are medium level. They do require action of some sort ($ \geq lowest\ priority$), but the action is due in the near future ($\leq highest\ priority$). I tend to skim over these emails…
Manipulating Chess-GPT’s World Model
An introduction to learned image compression using neural networks.
Searching Google Scholar for "certainly, here is" turns up a huge number of academic papers that include parts that were evidently written by ChatGPT - sections that start with "Certainly, …
[Hacker News discussion, LinkedIn discussion, Twitter thread]
Left SDXL output with SuperPrompt applied to the same input prompt.
This tutorial aims to give a gentle introduction to diffusion models, with a running example to illustrate how to build, train and sample from a simple diffusion model from scratch.
Introducing a series on AI vs. ads
Chronicles of training strong LLMs from scratch in the wild
AI, science, education, & ethics
Marketing, solicitations, advertising, political contact, direct mailers, email campaigns, cookies, ads — it's all opt out. It puts the burden on you to opt out of each communication each message, each outreach from a loyalty program. It can feel like weeding a garden, but that garden is your attention. The weeds keep…
Physics likes optimization! Subject to its boundary conditions, the time evolution of a physical system is a critical point for a quantity called an action. This point of view sets the stage for Noether's principle , a remarkable correspondence between continuous invariances of the action and conservation laws of the…
Nobody wants “temperamental and mysterious” products, so what will AI-integrated software development look like?
What are ML artifacts?
I think I’m probably going to lose quite a lot of money in the next year or two. It’s partly AI’s fault, but not mostly. Nonetheless I’m mostly going to write about AI, because it intersects the technosphere, where I’ve lived for decades.
Today, I’m going to write a bit about our GPU cluster machine we call cogito at Driveline Baseball. I’m going to timebox these blog posts so I get them out and also save my sanity, so t…
Corruption everywhere, even in YouTube's kids content