1
Summary of the 1.1 Billion Taxi Rides Benchmarks
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
Most cited
Posts that commenters keep linking to, ranked by how many different people have cited them in Hacker News comments. 27 posts.
2
3
Apache Arrow and the “Things I Hate About Pandas”
This post is the first of many to come on Apache Arrow, pandas, pandas2, and the general trajectory of my work in recent times and into the foreseeable future. This is a bit of a read and overall fairly technical, but if interested I encourage you to take the time to work through it.
4
The internals of PostgreSQL
After a temporary stay in Japan to focus on my health and personal responsibilities, I am fully recovered, ready, and excited to work abroad again.
5
Implementing Stripe-Like Idempotency Keys in Postgres
Building resilient services by identifying foreign state mutations and grouping local changes into restartable atomic phases so that every request can be driven to completion.
6
7
1.1B Taxi Rides on Kdb+/q and 4 Xeon Phi CPUs
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
8
Postgres Job Queues and Failure by MVCC
How Postgres’ concurrency model coupled with long-lived transactions can degrade the performance of indexes on hot tables in your database.
9
Be Careful with UUID or GUID as Primary Keys
You can use of UUIDs as the primary key to avoid database scale problems. But should you? I propose an alternative.
10
Index Investing Makes Markets and Economies More Efficient (2016)
U.S. equity index funds have grown dramatically in recent decades, from a negligible $500MM in assets in the early 1980s to a staggering $4T today. The consensus view in the investment community is that this growth is unsustainable. Indexing, after all, is a form of free-riding, and a market can only support so many…
11
Restic Cryptography
tl;dr: this is not an audit and I take no responsibility for your backups, but I had a quick look at the crypto and I think I’m going to use restic for my personal backups.
12
Pev: Postgres ‘Explain’ Visualizer (2016)
Postgres Explain Visualizer (Pev) is a tool I wrote to make EXPLAIN output easier to grok. It creates a graphical representation of the query plan.
13
14
Three-Valued Logic
Making decision despite missing SQL data (NULL): How to avoid OR…IS NULL in the WHERE clause.
15
Transactionally Staged Job Drains in Postgres
Building a robust background worker system that leverages transactional isolation to never work a job too early, and transactional durability to never let one drop.
16
A/B test of banner ads vs. traffic
9 months of daily A/B-testing of Google AdSense banner ads on Gwern.net indicates banner ads decrease total traffic substantially, possibly due to spillover effects in reader engagement and resharing.
17
18
Modes, Medians and Means: A Unifying Perspective
Introduction / Warning Link to heading Any traditional introductory statistics course will teach students the definitions of modes, medians and means. But, because introductory courses can’t assume that students have much mathematical maturity, the close relationship between these three summary statistics can’t be…
19
The Marketing Behind MongoDB
Countless NoSQL databases competed to be the database of choice. MongoDB's marketing strategy helped it become the winner.
20
Most Female and Male Occupations Since 1950
The shifting majorities of the sexes in the workplace.
21
Redis Streams and the Unified Log
Building a log-based architecture that’s fast, efficient, and resilient on the new stream data structure in Redis.
22
Literate SQL
Complex queries are hard to read. Literate SQL is about writing queries that are as easy to read as a good novel.
23
Scaling Postgres with Read Replicas and Using WAL to Counter Stale Reads
Scaling out operation with read replicas and avoiding the downside of stale reads by observing replication progress.
24
25
Stream VByte: breaking new speed records for integer compression
In many information systems, we work with arrays of integers. For example, maybe you need to keep track of which records in a database contain a given value. This sort of mapping can be expressed as an array of integers. These arrays can end up taking up a large fraction of your memory or disk … Continue reading…
26
Implementing faceted search with Django and PostgreSQL
I’ve added a faceted search engine to this blog, powered by PostgreSQL. It supports regular text search (proper search, not just SQL“like” queries), filter by tag, filter by date, filter …
27
Why Did So Many Startups Choose MongoDB?
NoSQL databases were the future. MongoDB was the database for “modern” web engineers and used by countless startups. What happened?