Writing an LLM from scratch, part 13 – attention heads are dumb
A pause to take stock: realising that attention heads are simpler than I thought explained why we do the calculations we do.
Topics /
The most popular blog posts about ai & machine learning on Hacker News. 1,227 posts.
A pause to take stock: realising that attention heads are simpler than I thought explained why we do the calculations we do.
After a long week, the Eye Security Research Team thought it was time for a small light-hearted distraction. So we rooted Copilot.
A new web search API is now available in Ollama. Ollama provides a generous free tier of web searches for individuals to use, and higher rate limits are available via Ollama’s cloud.
OpenAI recently released their open-weights model. Here we'll discuss how that inevitably leaks some information about their model training stack, and, on the way, show that GPT-5 was trained on phrases from adult websites....
Tor Browser 15.0a4 is now available from the Tor Browser download page and also from our distribution directory.
You can now generate videos in Gemini, powered by Veo 2.
Human drivers keep crashing into Waymos that aren't even moving.
I’ve been trying to slowly ease into using LLMs for coding help again lately (after quitting ), but something always feels off -- like we’re not quite on the...
How LLM-powered programming tools amplify developer capabilities rather than replace them
I’ve been reading this week about how humans learn, and effective ways of transferring knowledge. In addition, I’ve also had AI in the back of my mind, and...
Reproducibility is a bedrock of scientific progress. However, it’s remarkably difficult to get reproducible results out of large language models. For example, you might observe that asking ChatGPT the same question multiple times provides different results. This by itself is not surprising, since getting a result from…
A lot of people think LLMs are expensive to operate. That was true a couple of years ago, but these people haven't updated their views after an approximately 1000x reduction in prices over two years.
Also: another paper that seals the deal
AI coding tools boost productivity but may sacrifice the flow state and deep satisfaction developers experience when writing code by hand. What are we losing?
Despite the hype, Model Context Protocol (MCP) isn’t magic or revolutionary. But, it’s simple, well-timed, and well-executed. At Stainless, we’re betting it’s here to stay.
Jules is officially out of beta and launching publicly, powered by Gemini 2.5.During the beta, thousands of developers tackled tens of thousands of tasks, resulting in o…
Historically, general approaches always win in AI. Founders in AI application space now repeat the mistakes of AI researchers.
Notes from my colleagues on AI-assisted software delivery
A blog about making culture. Since 1999.
Although it might seem that AI will make it pointless, I still think it's worth blogging.
blog - kade killary
A series of foundation models for transparent AI in Europe
Cerebras powers the world's fastest AI inference on the biggest wafer chip. Cerebras CS-4 delivers up to 30x faster inference than GPUs.
OR: Claude will you go to prom with me?
Released last night, Grok 4 is now available via both API and a paid subscription for end-users. Update: If you ask it about controversial topics it will sometimes search X …
Why is DeepSeek-V3 supposedly fast and cheap to serve at scale, but too slow and expensive to run locally? Why are some AI models slow to respond but fast once…
Learn more about NotebookLM’s Audio Overviews feature and its expansion to more than 50 languages.
If search is the game, looks matter