151
RIP, Harvest
I’ve been a self-employed contractor for about a decade now, and I’m bummed about Harvest .
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 1,888 posts.
152
Writing a SQL database from scratch in Go
Writing a SQL database from scratch in Go: 1. SELECT, INSERT, CREATE and a REPL
153
Excel Never Dies
The Spreadsheet That Launched A Million Companies
154
Why databases use ordered indexes but programming uses hash tables
I think it is safe to state that hash tables (e.g. maps in Go, dicts in Python, HashMap in Java, etc.) are far more common than ordered data structures such as trees for in-memory data structures. One data point is that in a talk about C++ hash table optimization at Google ( available in Abseil ), the speaker noted…
155
156
100k TPS over a billion rows: the unreasonable effectiveness of SQLite
A blog mostly about Clojure programming
157
158
The 12-bit rainbow palette
A palette of twelve colours chosen with consideration for how we perceive luminance, chroma, and hue
159
Every NFL play for the past 10 years in CSV format
I've recently completed a project to compile publicly-available NFL play-by-play data. It took a while, but now it's ready. The resulting ...
160
161
PostgreSQL: Jsonb has committed
Great news: PostgreSQL introduced jsonb support, which is real data type with binary storage and indexing ! This work is a confluence of two projects - our hstore and json, which were existed in PostgreSQL for years in somewhat incomplete forms - hstore (10 years old!) implements key-value storage…
162
A humble guide to database schema design
Having a good database schema design is crucial for building applications that will need to scale in traffic and complexity. If you happen to do a bad design choice you will see that it takes a lot of effort to stop the pattern from propagating to your backends’ services and controllers, and finally to your frontend.
163
ZFS for Dummies
As mentioned on previous posts, I have spent the past few weeks dealing with a ZFS crash on my FreeNAS install. Because of that, not only was I forced to learn how to troubleshoot ZFS, but I also had to learn how to setup new volumes and come up with new backup strategies (between a few other things).
164
My finetuned models beat OpenAI's GPT-4
Finetunes of Mistral, Llama3 and Solar LLMs are more accurate for my test data than OpenAI’s models.
165
DuckDB Doesn't Need Data to Be a Database
One of the many enjoyable things about databases is that they generally try to separate how data is represented internally (say on disk) from how it is used. To the point that it has become the norm to not even store the data on the same hardware that is running the queries.
166
167
CRDTs: The Hard Parts [video]
A talk at Hydra distributed computing conference , online (originally planned to be in Moscow, Russia), 06 Jul 2020
168
7 Databases in 7 Weeks for 2025
7 databases to explore in 2025.
169
Is PostgreSQL good enough?
tldr; you can do jobs, queues, real time change feeds, time series, object store, document store, full text search with PostgreSQL. How to,...
170
Your data model is your destiny
Your product's core abstractions determine whether new features compound into a moat or just add to a feature list. Here's how to get it right.
171
How Google Maps allocates survival across London's restaurants
I wanted a dinner recommendation and got a research agenda instead. Using 13000+ restaurants, I rebuild its ratings with machine learning and map how algorithmic visibility actually distributes power.
172
173
A Visual Introduction to Machine Learning (2015)
What is machine learning? See how it works with our animated data visualization.
174
FireDucks: Pandas but Faster
Introduction My main background is a hedge fund professional, so I deal with finance data all the time and so far the Pandas library has been an indispensab...
175
It’s Time to Move on from Two Phase Commit
The two-phase commit protocol (2PC) has been used in enterprise software systems for over three decades . It has been an an incredibly impa...
176
New JSON query operators in SQLite 3.38.0
SQLite 3.38.0 introduced improvements to JSON query syntax using -> and ->> operators that are similar to PostgreSQL JSON functions. In this post we...
177
Period tracking app, Flo, found to be selling user data to Meta
What does this vagueness by-design mean for how we choose to self monitor our biological markers?
178
Let's build a full-text search engine
Full-Text Search is one of those tools people use every day without realizing it. If you ever googled "golang coverage report" or tried to find "indoor wireless camera" on an e-commerce website, you used some kind of full-text search.
179
Apache Arrow and the “Things I Hate About Pandas”
This post is the first of many to come on Apache Arrow, pandas, pandas2, and the general trajectory of my work in recent times and into the foreseeable future. This is a bit of a read and overall fairly technical, but if interested I encourage you to take the time to work through it.
180
Twelve years of distributed SQLite
In September 2014 I published a short post announcing rqlite. The idea was simple: take SQLite, put a Raft log in front of it, and replicate every write to a cluster so that the loss of a node did not mean the loss of the data. The source went on GitHub that same weekend. As…