1
SQLite small blob storage: 35% Faster Than the Filesystem
SQLite reads and writes small blobs (for example, thumbnail images) 35% faster¹ than the same blobs can be read from or written to individual files on disk using fread() or fwrite().
Most cited
Posts that commenters keep linking to, ranked by how many different people have cited them in Hacker News comments. 53 posts.
2
Summary of the 1.1 Billion Taxi Rides Benchmarks
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
3
Turning the database inside-out with Apache Samza (2015)
Confluent is building the foundational platform for data in motion so any organization can innovate and win in a digital-first world.
4
5
Hard Drive Cost per Gigabyte
For hard drive prices, the race to zero is over: nobody won. As you’ll see, the hard drive pricing curve has flattened out.
6
Postmortem of database outage of January 31
Postmortem on the database outage of January 31 2017 with the lessons we learned.
7
Spanner: TrueTime and external consistency
Understand Spanner's TrueTime and external consistency, ensuring globally consistent reads and writes across the distributed database.
8
Apache Arrow and the “Things I Hate About Pandas”
This post is the first of many to come on Apache Arrow, pandas, pandas2, and the general trajectory of my work in recent times and into the foreseeable future. This is a bit of a read and overall fairly technical, but if interested I encourage you to take the time to work through it.
9
Jepsen: CockroachDB beta-20160829
CockroachDB is a distributed SQL database, loosely patterned after Google Spanner and designed for semi-synchronous networks. It speaks the PostgreSQL wire protocol , supports a reasonable dialect of SQL , and combines replication for durability with transparent scale-out sharding for large tables. Its most…
10
MongoDB 3.4.0-rc3
In the past year and a half, MongoDB has put a good deal of work into improved read safety, enabled by the adoption of their new replication protocol and the WiredTiger storage engine. Dirty reads were addressed in 3.2 by introducing a majority read concern , and stale reads were addressed with the introduction of the…
11
The internals of PostgreSQL
After a temporary stay in Japan to focus on my health and personal responsibilities, I am fully recovered, ready, and excited to work abroad again.
12
Exactly-once Semantics: How Kafka Does it
Exactly-once is a hard problem to solve, but we've done it. Available now in Apache Kafka 0.11, exactly-once semantics.
13
Ways to paginate in Postgres (2016)
Pagination, pervasive as it is in web applications, is easy to implement inefficiently. This post examines different methods of server-side pagination and their tradeoffs in PostgreSQL. Methods explored include limit-offset, cursors, keyset pagination, as well as more exotic techniques.
14
Inside Cloud Spanner and the CAP Theorem
Building systems that manage globally distributed data, provide data consistency and are also highly available is really hard. The beauty of the cloud is that someone else can build that for you.
15
Implementing Stripe-Like Idempotency Keys in Postgres
Building resilient services by identifying foreign state mutations and grouping local changes into restartable atomic phases so that every request can be driven to completion.
16
17
How Discord Stores Billions of Messages Using Cassandra
Discover the latest news and app updates, stories, company insights, and plenty more about Discord, all in one place.
18
1.1B Taxi Rides on Kdb+/q and 4 Xeon Phi CPUs
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
19
20
Postgres Job Queues and Failure by MVCC
How Postgres’ concurrency model coupled with long-lived transactions can degrade the performance of indexes on hot tables in your database.
21
Be Careful with UUID or GUID as Primary Keys
You can use of UUIDs as the primary key to avoid database scale problems. But should you? I propose an alternative.
22
23
Index Investing Makes Markets and Economies More Efficient (2016)
U.S. equity index funds have grown dramatically in recent decades, from a negligible $500MM in assets in the early 1980s to a staggering $4T today. The consensus view in the investment community is that this growth is unsustainable. Indexing, after all, is a form of free-riding, and a market can only support so many…
24
Restic Cryptography
tl;dr: this is not an audit and I take no responsibility for your backups, but I had a quick look at the crypto and I think I’m going to use restic for my personal backups.
25
RethinkDB versus PostgreSQL: my personal experience
News and updates about CoCalc
26
Pev: Postgres ‘Explain’ Visualizer (2016)
Postgres Explain Visualizer (Pev) is a tool I wrote to make EXPLAIN output easier to grok. It creates a graphical representation of the query plan.
27
28
Delivering Billions of Messages Exactly Once
The single requirement of all data pipelines is that they cannot lose data. Data can usually be delayed or re-ordered–but never dropped.
29
Creating a Modern OCR Pipeline Using Computer Vision and Deep Learning
Dash knows your context, your team, and your work, so your team can stay organized, easily find and share knowledge, and keep projects secure, all from one place.
30
Three-Valued Logic
Making decision despite missing SQL data (NULL): How to avoid OR…IS NULL in the WHERE clause.