1
Database Fundamentals
The fundamental problems a database engineer thinks about in the shower
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 159 posts.
2
Choose Postgres queue technology
Introduction Postgres queue tech is a thing of beauty, but far from mainstream. Its relative obscurity is partially attributable to the cargo cult of “scale”. The scalability cult has decreed that there are several queue technologies with greater “scalability” than Postgres, and for that reason alone, Postgres isn’t…
3
Ways to shoot yourself in the foot with Postgres
Personal website of Phil Booth
4
Embeddings: What they are and why they matter
Embeddings are a really neat trick that often come wrapped in a pile of intimidating jargon. If you can make it through that jargon, they unlock powerful and exciting techniques …
5
6
Postgres as a graph database
You don't need Neo4j for graph data. Walking through nodes, edges, and recursive traversals in plain Postgres, with working SQL.
7
Most data work seems fundamentally worthless
After going viral, there is one comment I'd like to address. One guy pointed out how comfortable this work seemed, how high the pay was, and said I am out-of-touch and privileged. To that I say, yes, obviously . I moved to the first-world to have first-world problems. Mission accomplished.
8
9
Retrieving 1TB of data from a faulty drive with the help of woodworking tools
So, a while ago I built a gaming PC with the following specification: Case Dan A4 v4.1 Motherboard Z690I Strix Gaming Pro...
10
Joins 13 Ways
Relational (inner) joins are really common in the world of databases, and one weird thing about them is that it seems like everyone has a different idea of what they are. In this post I’ve aggregated a bunch of different definitions, ways of thinking about them, and ways of implementing them that will hopefully be…
11
SQL should be the default choice for data transformation logic
SQL should be the first option considered for new data engineering work. It’s robust, fast, future-proof and testable. With a bit of care, it’s clear and readable.
12
ZFS for Dummies
As mentioned on previous posts, I have spent the past few weeks dealing with a ZFS crash on my FreeNAS install. Because of that, not only was I forced to learn how to troubleshoot ZFS, but I also had to learn how to setup new volumes and come up with new backup strategies (between a few other things).
13
Bad numbers in the “gzip beats BERT” paper?
The recent paper, “Low-Resource” Text Classification: A Parameter-Free Classification Method with Compressors by Jiang et al. [ link ] recently received a lot of attention on twitter.
14
Bullshit graph database performance benchmarks
Hey HackerNews, let me just drop my mixtape, checkout my soundcloud and “Death Row” is the label that pays me. How is the Graph Database category supposed to grow when vendors keep spou…
15
Algorithmic Trading: A Practitioner’s Guide
I really enjoyed reading Algorithmic Trading: A Practitioner’s Guide by Jeffrey M. Bacidore. Before starting, I imagined it would cover various strategies for trading in the markets, along th…
16
Text Editor Data Structures
Text editors can be an interesting challenge to program. The types of problems that text editors need to solve can range from trivial to mind-bogglingly difficult. Recently, I have been on something of a spiritual journey to rework some internal data structures in an editor I have been building, specifically the most…
17
How to make fancy road trip maps with R and OpenStreetMap
Use R to get geocoded location and routing data from OpenStreetMap and explore our family’s impending 5,000 mile road trip around the USA
18
River: A fast, robust job queue for Go and Postgres
Introducing River, a Postgres-based job queue designed for resilience and correctness through strong transactional guarantees.
19
Tracking SQLite Database Changes in Git
Note : This post stirred up some interesting discussions on HackerNews and Lobste.rs .
20
Forecasts need to have error bars
Richard Rosenfeld in the most recent Criminologist published a piece about forecasting national level crime rates. People complain about the FBI releasing crime stats a year late, academics are wor…
21
I don't need your query language
Seriously, I don't. I'd prefer SQL.
22
50 years later, is two-phase locking the best we can do?
Two phase locking (2PL) was the first of the general-purpose Concurrency Controls to be invented which provided Serializability. In fact, 2P...
23
The end of a myth: Distributed transactions can scale
This paper appeared in VLDB'17. The paper presents NAM-DB, a scalable distributed database system that uses RDMA (mostly 1-way RDMA) and a ...
24
25
Fine, I'll run a regression analysis but it won't make you happy
State partisanship and COVID vaccination rates are strongly predictive of COVID death rates even once you account for age.
26
27
Understanding Parquet, Iceberg and Data Lakehouses
In the past few years, I've heard a lot about Avro, Parquet, ORC, Arrow and Feather, but I also keep hearing about Iceberg and Delta Lake. As a "database person", I’ve been struggling to understand all of these different things, and how they relate to Data Lakes and
28
Pushing ChatGPT's Structured Data Support to Its Limits
“Function calling” with ChatGPT is ChatGPT’s best feature since ChatGPT.
29
I downloaded all 1.6M posts on Bluesky
I downloaded all the posts on Bluesky as of 2023-05-01. Then I did some data analysis on the 1680399 posts from 45457 accounts.
30
Cola: A text CRDT for real-time collaborative editing
A deep dive into the theoretical framework and the technical implementation of a text CRDT.