91
Modeling Starlink Capacity
Prior attempts to calculate Starlink’s potential for service have been theoretical. We simulate based on the actual constellation.
Most cited
Posts that commenters keep linking to, ranked by how many different people have cited them in Hacker News comments. 253 posts.
92
93
94
Postgres Job Queues and Failure by MVCC
How Postgres’ concurrency model coupled with long-lived transactions can degrade the performance of indexes on hot tables in your database.
95
Be Careful with UUID or GUID as Primary Keys
You can use of UUIDs as the primary key to avoid database scale problems. But should you? I propose an alternative.
96
97
Xkcd-Style Plots in Matplotlib (2012)
Update: the matplotlib pull request has been merged! See This post for a description of the XKCD functionality now built-in to matplotlib!
98
Why xkcd-style graphs are important
For most data scientists, a toolkit like scipy/matplotlib or R becomes so familiar that it becomes almost an extension of their own mind. When fleshing out an idea, we will just open up ipython, write a few lines of code, and display the plot on the screen to get …
99
100
101
Damn Cool Algorithms: Cardinality Estimation
Posted by Nick Johnson | Filed under python , cardinality-estimation , damn-cool-algorithms
102
103
Using MySQL as a NoSQL - A story for exceeding 750,000 qps on a commodity server
UPDATE: Oracle officially released memcached daemon plugin that talks with InnoDB. I'm glad to see that NoSQL+MySQL has become an official ...
104
Jepsen: RethinkDB 2.2.3 reconfiguration
In the previous Jepsen analysis of RethinkDB , we tested single-document reads, writes, and conditional writes, under network partitions and process pauses. RethinkDB did not exhibit any nonlinearizable histories in those tests. However, testing with more aggressive failure modes, on both 2.1.5 and 2.2.3, has…
105
Index Investing Makes Markets and Economies More Efficient (2016)
U.S. equity index funds have grown dramatically in recent decades, from a negligible $500MM in assets in the early 1980s to a staggering $4T today. The consensus view in the investment community is that this growth is unsustainable. Indexing, after all, is a form of free-riding, and a market can only support so many…
106
Recommending music on Spotify with deep learning
An overview of what I've been doing as part of my internship at Spotify in NYC this summer: using convolutional neural networks for audio-based music recommendation.
107
Gay marriage: the database engineering perspective
Update, 2017-09-25 Hi! This essay is undergoing a long-overdue rewrite. In the meantime, consider my 2015 follow-up essay, Obergefell v. Hodges: the database engineering perspective....
108
Simple declarative schema migration for SQLite
At my company we use a small SQLite database. We define the schema in a single file with SQL CREATE TABLE statements. If we add tables, columns, or indexes, our application will create them automatically the next time it starts up.
109
Restic Cryptography
tl;dr: this is not an audit and I take no responsibility for your backups, but I had a quick look at the crypto and I think I’m going to use restic for my personal backups.
110
One-liner for running queries against CSV files with SQLite
I figured out how to run a SQL query directly against a CSV file using the sqlite3 command-line utility:
111
Soft deletion probably isn't worth it
The traditional soft deletion pattern using <code>deleted_at</code> columns leaks into code, curbs the usefulness of foreign keys, and makes data removal difficult. Luckily, there’s an alternative.
112
Pev: Postgres ‘Explain’ Visualizer (2016)
Postgres Explain Visualizer (Pev) is a tool I wrote to make EXPLAIN output easier to grok. It creates a graphical representation of the query plan.
113
114
PostgreSQL Scalability: Towards Millions TPS
PostgreSQL scalability on multicore and multisocket machines became a subject of optimization long time ago once such machines became widely used. This blog post shows brief history of vertical scalability improvements between versions 8.0 and 8.4. PostgreSQL 9.2 had very noticeable scalability improvement. Thanks to…
115
NewSQL databases fail to guarantee consistency and I blame Spanner
(Spanner vs. Calvin, Part 2) [TL;DR I wrote a post in 2017 that discussed Spanner vs. Calvin that focused on performance differences. Th...
116
Personal Data Warehouses: Reclaiming Your Data
I gave a talk yesterday about personal data warehouses for GitHub’s OCTO Speaker Series, focusing on my Datasette and Dogsheep projects. The video of the talk is now available, and …
117
Consider SQLite
If you were creating a web app from scratch today, what database would you use? Probably the most frequent answer I see to this is Postgres, although there are a wide range of common answers: MySQL, MariaDB, Microsoft SQL Server, MongoDB, etc. Today I want you to consider: what if SQLite would do just fine?
118
119
Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS
Today is a very exciting day as we release Amazon DynamoDB, a fast, highly reliable and cost-effective NoSQL database service designed for internet scale applications. DynamoDB is the result of 15 years of learning in the areas of large scale non-relational databases and cloud services. Several years ago we published…
120
EBCDIC is incompatible with GDPR
Welcome to acronym city! The Court of Appeal of Brussels has made an interesting ruling. A customer complained that their bank was spelling the customer's name incorrectly. The bank didn't have support for diacritical marks. Things like á, è, ô, ü, ç etc. Those accents are common in many languages. So it was a little…