91
A SQL Heuristic: ORs Are Expensive
But this is slow! With 1,000,000 applications and 1000 users uniformly distributed on both columns, it takes over 100ms. 1
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 148 posts.
92
93
Absurd Workflows: Durable Execution with Just Postgres
Durable execution with just postgres.
94
A data analysis of speeches at the Oscars
I analysed almost 2,000 Oscar speeches to discover if the claim that Harvey Weinstein was thanked more often than God is true. Plus, we'll find out which Hollywood icon is bigger than both of them.
95
Charting Form Ds to roughly see the state of venture capital “fund” raising
Above is a graph that displays the amount of Form Ds filed, where the entity (read: company/firm) name contains the phrases "fund I", "fund II", "fund III", and "fund IV". The x-axis is not the prettiest, but it is broken down by quarter. You can see that the line for "fund I" sees by far the greatest peak around…
96
I analyzed the lineups at the most popular nightclubs
A few years back I did a bit of dance music related data visualization over at Lazily Evaluated. My favourite was an analysis of clubs and their lineups using Resident Advisor / RA data, I called it Clubster Analysis. I always wanted to dig into the technical aspects of gathering the data, analyzing it and building…
97
Pipelining in psql (PostgreSQL 18)
the psql client version 18 comes with pipelining, which can speed up client-server communication. In this post, let's see how it works and how much can be g...
98
Strong Eventual Consistency – The Big Idea Behind CRDTs
Missing the Forest for the Sequence Trees.
99
Building a high-performance ticketing system with TigerBeetle
Rene Schallner (@renerocksai): software engineer turned AI researcher and engineer, in Vienna.
100
DSM Disorders Disappear in Statistical Clustering of Psychiatric Symptoms (2024)
Whither Major Depression?
101
Mapping the off-target effects of every FDA-approved drug in existence
6.2k words, 29 minutes reading time
102
Supercharge SQLite with Ruby Functions
An interesting twist in my recent usage of SQLite was the fact that I noticed my research scripts and the database intertwine more. SQLite is unique in that it really lives in-process, unlike standalone database servers. There is a feature to that which does not get used very frequently, but can be indispensable in…
103
League of Legends data scraping the hard and tedious way for fun
Personal website for some random tidbits I work on
104
Hachi: An Image Search Engine
For quite some time now, i have been working on and off on a fully self-hosted search engine , in hope to make it easier to search across Personal data in an end to end manner. Even as individuals, we are hoarding and generating more and more data with no end in sight. Such "personal" data is being stored from local…
105
You probably don't need query builders
In a previous [0] post I’ve discussed why I ditched sea-query and why sql is almostalways the best way to not re-learn something new from the beginning...
106
RegreSQL: Regression Testing for PostgreSQL Queries
Stop deploying broken SQL queries. RegreSQL provides regression testing for PostgreSQL queries with performance baselines and automated warnings.
107
Zmij: Faster floating point double-to-string conversion
There comes a time in every software engineer’s life when they come up with a new binary-to-decimal floating-point conversion method. I guess my time has come. I just wrote one, mostly over a weekend: https://github.com/vitaut/zmij .
108
SQL needed structure
The data on the page is presented in a hierarchichal structure. The movie page contains a director, a list of genres, a list of actors, and each actor in the list contains a list of characters they played in the movie. You can't sensibly fit all of this into a single flat structure like a relation.
109
What Does a Database for SSDs Look Like?
What does a relational database designed specifically for local SSDs look like? Postgres, MySQL, SQLite and many others were invented in the 90s and 00s, the era of spinning disks. A local NVMe SSD has ~1000x improvement in both throughput and latency. Design decisions like write-ahead logs, large page sizes, and…
110
The World's 2.75B Buildings
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
111
Street address errors in Google Maps
(or actually they do, but they don’t use this knowledge effectively)Update, April 26, 2025: the address fix for W 6th Ave is live, mostly. Going forward I wish that Google Maps would make it harder…
112
113
SQLite's documentation about its durability properties is unclear
Is SQLite durable by default? What settings guarantee durability? The documentation and even comments from its creator give conflicting answers.
114
I made a search engine worse than Elasticsearch (2024)
Integrating my BM25 pandas search library, SearchArray, into BEIR, in order to embarass myself in public.
115
Losing a 5-year-long Illinois FOIA lawsuit for database schemas
Thomas Ptacek, a friend and expert witness in this lawsuit summed it up best in the court's hallway while walking within three feet of opposing counsel: "This is fucking stupid" .
116
Subtleties of SQLite Indexes
Understanding query planner quirks yielded a ~35% speedup.
117
How exchanges turn order books into distributed logs
Every modern exchange is a distributed database in disguise. This article reveals how trading engines transform chaotic streams of buy and sell orders into a perfectly ordered, replayable log, ensuring fairness, determinism, and market data reliability.
118
Build your own SQLite, Part 4: reading tables metadata
As we saw in the opening post, SQLite stores metadata about tables in a special "schema table" starting on page 1. We've been reading records from this…
119
Loading Pydantic models from JSON without running out of memory
Pydantic’s JSON loading uses a huge amount of memory; here’s how to reduce it.
120
Just make it scale: An Aurora DSQL story
AWS Senior Principal Engineers, Niko Matsakis and Marc Bowes, take us inside Aurora DSQL's development: scaling write operations without two-phase commit, overcoming garbage collection hurdles, and embracing Rust for both data and control planes.