AI agent benchmarks are broken
Benchmarks are foundational to evaluating the strengths and limitations of AI systems, guiding both research and industry development.
Topics /
The most popular blog posts about ai & machine learning on Hacker News. 3,091 posts.
Benchmarks are foundational to evaluating the strengths and limitations of AI systems, guiding both research and industry development.
MCPs—Model Context Protocols—are set to transform AI from passive chatbots into powerful, action-taking agents. But the real story isn’t what MCPs enable—it’s who controls them.
Give any AI agent a persistent memory in minutes. Works with Claude, ChatGPT, Ollama, OpenRouter, and any MCP-compatible agent. Open source, self-hosted, model-agnostic.
Daily, snackable writings to spur changes in thinking. Building a blueprint for a better brain by tinkering with the code.
A little bit of magic, but mostly just practice
That doesn’t spy on you
For Pycon 2025, I created a poster exploring vector embedding models, which you can download at full-size . In this post, I'll translate ...
When I attended NAACL, I wanted to do a little test. I had two pitches for my LLM.int8() paper. One pitch is about how I use advanced quantization methods to achieve no performance degradation transformer inference at scale that makes large models more accessible. The other pitch talks about emergent outliers in…
The other day I wanted to look up a specific IBM PS/2 model, a circa 1992 PS/2 Server system. So I punched the model into Google, and got this:
Slopforks: what happens when a library gets rewritten with AI?
[UPDATE: Feb 8 2025] my amazing colleague Max Shen noticed a sign mistake in my derivation of the partial derivative of the log-harmonic function below.
I like HackerNews, but I don't love that so much of it has turned into discussion of a single topic: AI. This is a version of HackerNews, filtered to remove any article focusing on __AI__. Refreshes about every ten minutes.
Bayesian statistics rely heavily on Monte-Carlo methods. A common question that arises is “isn’t there an easier, analytical solution?” This post explores a bit more why this is by breaking down the analysis of a Bayesian A/B test and showing how tricky the analytical path is and exploring more of t
Don't let AI erode your focus.
Taking the output of many dumb LLM search relevance judges and feeding the output to a decision tree to improve precision
From zero to linear regression
Since writing this post back in 2018, I have extended this to a 4-part series on causal inference: * ➡️️ Part 1: Intro to causal inference and do-calculus [https://www.inference.vc/untitled] * Part 2: Illustrating Interventions with a Toy Example…
I explain what is so unique about the RWKV language model.
In this post we’ll explore how we can derive logistic regression from Bayes’ Theorem. Starting with Bayes’ Theorem we’ll work our way to computing the log odds of our problem and the arrive at the inverse logit function. After reading this post you’ll have a much stronger intuition for how logistic
Deep code search Gu et al., ICSE’18 The problem with searching for code is that the query, e.g. “read an object from xml,” doesn’t look very much like the source code snippets tha…
If you ever tried to learn how the Harris corner detection algorithm works, you might have noticed that the process is not intuitive at all. First, you start with an energy function, approximate it using Taylor approximation, get a matrix from that, then find the eigenvalues of that matrix, etc. But when you come to…
Many new technologies have unexpected impacts on the physical or social world in which we live.
My input stream is full of it: Fear and loathing and cheerleading and prognosticating on what generative AI means and whether it’s Good or Bad and what we should be doing. All the channels: Blogs and peer-reviewed papers and social-media posts and business-news stories. So there’s lots of AI angst out there, but this…
The 2019 paper “Quantum supremacy using a programmable superconducting processor” asserted that Google’s Sycamore quantum computer, with 53 qubits and a depth of 20, performed a specific computat…
Alex Moon is an Australian-British software engineer and artist based in the North East of England.
Cosine similarity measures the angle between two vectors — the dot product divided by the product of their magnitudes. Here's what each part of the formula does.