31
What Is Entropy?
I wrote a little book about entropy; here’s the current draft: • What is Entropy? If you see typos and other mistakes, or have trouble understanding things, please let me know! An altern…
Topics /
Math & Algorithms
The most popular blog posts about math & algorithms on Hacker News. 237 posts.
32
Sort, sweep, and prune: Collision detection algorithms (2023)
Sweep-and-prune is my go-to algorithm when I want to quickly implement collision detection for a game. I think it’s an awesome and elegant algorithm, so I wrote a post about it.
33
Alonzo Church: Architect of computer intelligence
The man who gave us the solution to the Entscheidungsproblem
34
Neural network training makes beautiful fractals
This blog is intended to be a place to share ideas and results that are too weird, incomplete, or off-topic to turn into an academic paper, but that I think may be important. Let me know what you think! Contact links to the left.
35
Diffusion Models
I spent 2022 learning to draw and was blindsided by the rise of AI art There is lots to say about this, whether it has been or will be a good thing for artists and society in the long run. I hope to write about it in another post. models like Stable Diffusion. Suddenly, the computer was a better artist than I could…
36
Bayesian Statistics: The three cultures
A traditional subjective Bayesian will assume a data generating distribution (aka a likelihood when viewed as a function of the parameters). Then, conditional on this assumption, they encode their prior beliefs as a prior distribution over parameters. After this, they turn the crank on posterior inference and don’t…
37
The Truth About Linear Regression (2015)
This is basically a compilation of the lecture notes I wrote when teaching 36-401, Modern Regression, in fall 2015 . I offer it here on the chance that it might be of interest to those learning, or teaching, linear regression. There's no shortage of resources on that, but I have tried to present the subject as though…
38
39
BB(3, 4) > Ack(14)
Pavel has found a new 3-state 4-symbol Busy Beaver champion which can compute an “Ackermann-level” function and halts with exactly
40
41
Where is Noether's principle in machine learning?
Physics likes optimization! Subject to its boundary conditions, the time evolution of a physical system is a critical point for a quantity called an action. This point of view sets the stage for Noether's principle , a remarkable correspondence between continuous invariances of the action and conservation laws of the…
42
43
A visual interactive guide to Bloom filters
A visual, interactive guide to what bloom filters are, when you would use them, and how they work.
44
The simple beauty of XOR floating point compression
I recently implemented a small program to visualize the inner workings of a scheme that compresses floating point timeseries by XORing subsequent values. The resulting visualizations are quite neat…
45
Exponentially Better Rotations (2022)
If you’ve done any 3D programming, you’ve likely encountered the zoo of techniques and representations used when working with 3D rotations. Some of them are better than others, depending on the situation.
46
Yes, we did discover the Higgs
Ben Recht got my attention with a tweet about his blog post "The Higgs discovery did not take place.” I’m generally a fan of Ben’s writing, but I think he’s off base on a number of issues, and I’ll explain why. [Updated Oct 23rd, 2024] Ben's …
47
How hard can generating 1024-bit primes be?
On the one hand they are easy to explain, they are just numbers that have no factors other than one and themselves, but on the other hand they contain endless complexity. They show up in numerous places, ranging from mathematical concepts and conjectures to interesting looking visualizations and cryptography…
48
Algorithms we develop software by
Pathfinding applied to the software solution domain
49
A quick post on Chen's algorithm
Update (April 19): Yilei Chen announced the discovery of a bug in the algorithm, which he does not know how to fix. This was independently discovered by Hongxun Wu and Thomas Vidick. At present, th…
50
What has a 1 in a million chance? (2010)
If you overheard the phrase "1 in a million chance" in someone else's casual conversation, what might they be talking about?
51
What I've learned about flow fields so far
Noise functions and the tiniest bit of trigonometry can take you a long way towards making art.
52
An RNG that runs in your brain
Humans are notoriously bad at coming up with random numbers. I wanted to be able to quickly generate “random enough” numbers. I’m not looking for anything that great, I just want to be able to come up with the random digits in half a minute. Some looking around brought me to an old usenet post by George Marsaglia…
53
Why Aren't We Sieve-Ing?
Long-time readers of this blog will know that I have mixed feelings about caches. One on hand, caching is critical to the performance of systems at every layer, from CPUs to storage to whole distributed architectures. On the other hand, caching being this critical means that designers need to carefully consider what…
54
Is the frequency domain a real place?
Dissing Fourier, exploring the Hadamard-Walch transform, and making square-wave spectrograms.
55
Category Theory Illustrated: Logic (2021)
Now let’s talk about one more seemingly unrelated topic just so we can “surprise” ourselves when we realize it’s category theory. By the way, in this chapter there will be another surprise in addition to that, so don’t fall asleep.
56
The zombie misconception of theoretical computer science
In Michael Sipser’s Introduction to the Theory of Computation textbook, he has one Platonically perfect homework exercise, so perfect that I can reconstruct it from memory despite not having …
57
AlphaProof's Greatest Hits
Here I’ll try to explain the coolest ideas in each of AlphaProof’s IMO 2024 solutions. AlphaProof produces proofs in Lean, and each Lean proof is composed of a series of tactics. So I’ll pick out the tactics that correspond to these ideas in the proofs for problems 1, 2 and 6 (the three problems that AlphaProof…
58
Counterintuitive Properties of High Dimensional Space (2018)
Our geometric intuition developed in our three-dimensional world often fails us in higher dimensions. Many properties of even simple objects, such as higher dimensional analogs of cubes and spheres, are very counterintuitive. Below we discuss just a few of these properties in an attempt to convey some of the weirdness…
59
Finding near-duplicates with Jaccard similarity and MinHash
How do you find near-duplicates in a massive collection of documents? An exploration of the Jaccard similarity metric, and the MinHash hashing trick used to efficiently approximate it at web scale.
60
Clean your codebase with basic information theory
Cut out everything that's not surprising.