Throughout the last year I’ve written in detail about the rot in tech — the spuriousness of charlatans looking to accumulate money and power, the desperation of the most powerful executives to maintain control and rapacious growth, and the speciousness of the latest hype cycle — but at the end of
Homepage of Tri Dao.
1) Foundational math. 2) Classical machine learning. 3) Deep learning. 4) Cutting-edge machine learning.
Left SDXL output with SuperPrompt applied to the same input prompt.
AI, science, education, & ethics
tl;dr; if you want to pass HTML data to GPT-4o, just strip out all the HTML and pass raw text, it’s cheaper and there is little to no performance degradation. Source code and demo available.
Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think
Last Updated: January 23, 2026There’s no question that AI will impact jobs. But which jobs are more likely…
... and how that provides winning evals
Introducing a series on AI vs. ads
This article explains a simple model of how minds might work. I’m motivated by the success of AI-based language models to look at the future of digital minds. I’ll present a conceptual data-flow architecture that can account for several key features of minds: the ability to initiate actions (agency), learning…
Over the past month I’ve been exploring the rapidly evolving world of Large Language Models (LLM). It’s now accessible enough to run a LLM on a Raspberry Pi smarter than the original ChatGPT (November 2022). A modest desktop or laptop supports even smarter AI. It’s also private, offline, unlimited, and…
There's been a fair bit of buzz about Kolmogorov-Arnold networks online lately. Some research papers were posted around claiming that they offer better accuracy or faster training compared to traditional neural networks/MLPs for the same parameter count.
A deep dive on Microsoft's GraphRAG paper found questionable metrics with vaguely defined lift, so I analyzed knowledge graphs in RAG overall using Neo4j vs FAISS
Important new study from Apple
Quickly understand inscrutable LLM frameworks by intercepting API calls.
This article will teach you about self-attention mechanisms used in transformer architectures and large language models (LLMs) such as GPT-4 and Llama.
I gave a talk about accessing Large Language Models from the command-line last week as part of the Mastering LLMs: A Conference For Developers & Data Scientists six week long …
are they good or bad?
I think we have an answer on whether AIs will reshape work....
This article provides a short introduction to the pipeline used to create the data to train large language models (LLMs) such as LLaMA using Common Crawl (CC).
I summarise the kinds of evaluations that are needed for a structured data generation task.
Protein “language” is a lot like human language. Given the similarities, researchers have been building and training language models on protein sequence data, replicating the success seen in other domains, with profound implications. In this post, I will explore how transformer models have been applied to protein data…
Matthew Honnibal is a computational linguist from Sydney based in Berlin, Germany. He's the author of the spaCy Natural Language Processing library and the co-founder of Explosion.
I had a really enlightening talk with Daniel Slutsky this week (who is an exceptional data scientist, software engineer, and community organizer I highly…
