We love TPUs at Google, but GPUs are great too. This chapter takes a deep dive into the world of GPUs – how each chip works, how they're networked together, and what that means for LLMs, especially compared to TPUs. While there are a multitude of GPU architectures from NVIDIA, AMD, Intel, and others, here we will…
Hatching... Cursor Shenanigans Cursor, my beloved, started rate limiting shenanigans a few days back. For a good 2 weeks after June 16, 2025, we had almo...
ChatGPT and LLMs amplify both insight and illusion. They are not knowledge engines, but confidence engines shaping how we think and learn.
Toyota Motor Corporation (“Toyota”) and Waymo reached a preliminary agreement to explore a collaboration focused on accelerating the development and deployment of autonomous driving technologies.
Our new flagship Olmo 3 model family empowers the open source community with not only state-of-the-art open models, but the entire model flow and full traceability back to training data.
How to build a RAG system that's fully self-hosted.
A developer's guide to downloading and running LLMs on macOS, for experimentation and privacy.
But for what I do use LLMs for, it’s invaluable.
Last month I asked people to hack part of my YouTube video, specifically to de-pixelate the contents of a folder I had pixelated starting at the 4:57 mark. Your browser does not support the video tag. For years, people have used the censor tool to blur or pixelate out parts of videos where there's sensitive…
Following the posting of the preprint paper “Artificial Intelligence, Scientific Discovery, and Product Innovation” on arXiv in November 2024, concerns were raised about the integrity of the research. MIT conducted an internal, confidential review and concluded that the paper should be withdrawn from public discourse.
The path to Vision Zero requires reducing severe crashes and improving the safety of those most at risk. Our latest research paper shows that the Waymo Driver is making significant strides in both areas. By reducing the most dangerous crashes and providing better protection for pedestrians, cyclists, and other…
We trained diffusion large language models that are up to 10x faster and cheaper than current LLMs, pushing the frontier of intelligence and speed for language models.
As everyone is well aware, the world is still going nuts trying to develop more, newer and better AI tools. Mostly by throwing absurd amounts of money at the problem. Many of those billions go towards building cheap or free services that operate at a significant loss. The tech giants that run them all are hoping to…
Analysis of DeepSeek-R1 censorship using 1,156 political prompts exposing CCP content filtering and bias detection patterns
One paper that caught my attention a bit ago was Selective attention in hypothesis-driven data analysis by Itai Yanai and Martin Lercher. In their study, students who were given specific hypotheses to test were much less likely to notice an obvious “gorilla in the data” compared to students who explored the data…
2025 Year in Review of LLM paradigm changes
A technical report on three bugs that intermittently degraded Claude's responses: what happened, why fixes took time, and what we're changing.
A TL;DR version of Andrej Karpathy's "Deep dive into LLMs like ChatGPT" video.
Moving on from a toy self-attention mechanism, it's time to find out how to build a real trainable one. Following Sebastian Raschka's book 'Build a Large Language Model (from Scratch)'. Part 8/??
We now live in a world where ML workflows (pre-training, post training, etc) are heterogeneous, must contend with hardware failures, are increasingly asynchronous and highly dynamic. Traditionally, PyTorch has relied...
A report describing a highly sophisticated AI-led cyberattack and how we disrupted it.
Continual learning is a huge bottleneck
How to use AI coding assistants without letting your hard-earned engineering skills wither away.
ML, Biotech, Hardware, and Coordination Problems. Sometimes I write about hard problems and how to solve them.
Today, we release FLUX.2, our most capable model to date.
A surprisingly common complaint I see from developers who have tried using LLMs for code is that they encountered a hallucination—usually the LLM inventing a method or even a full …
I believe that language models aren’t world models. It’s a weak claim — I’m not saying they’re useless, or that we’re done milking them. It’s also a fuzzy-sounding claim — with its trillion weights, who can prove that there’s something an LLM isn't a model of? But I hope to make my claim clear and persuasive enough…
It's been just over two years and two months since ChatGPT launched, and in that time we've seen Large Language Models (LLMs) blossom from a novel concept into one of the most craven cons of the 21st century — a cynical bubble inflated by OpenAI CEO Sam Altman built
