Transactionally Staged Job Drains in Postgres
Building a robust background worker system that leverages transactional isolation to never work a job too early, and transactional durability to never let one drop.
Posts that commenters keep linking to, ranked by how many different people have cited them in Hacker News comments. 53 posts.
Building a robust background worker system that leverages transactional isolation to never work a job too early, and transactional durability to never let one drop.
This post covers how Apache Kafka and its Streams API are used for storing and processing all the articles ever published by The New York Times
We take a deep dive on hard drive performance stats from 73,653 hard drives. What are the most reliable hard drives by size, manufacturer, and model?
9 months of daily A/B-testing of Google AdSense banner ads on Gwern.net indicates banner ads decrease total traffic substantially, possibly due to spillover effects in reader engagement and resharing.
Why graphical representation and visualization are so important to...
Late last year I told you about our plans to add PostgreSQL compatibility to Amazon Aurora. We launched the private beta shortly after that announcement, and followed it up earlier this year with an open preview. We’ve received lots of great feedback during the beta and the preview and have done our best to make […]
Launching CockroachDB 1.0, a production-ready release with distributed SQL and multi-active availability in two tiers: CockroachDB Core and CockroachDB Self-hosted.
Introduction / Warning Link to heading Any traditional introductory statistics course will teach students the definitions of modes, medians and means. But, because introductory courses can’t assume that students have much mathematical maturity, the close relationship between these three summary statistics can’t be…
Countless NoSQL databases competed to be the database of choice. MongoDB's marketing strategy helped it become the winner.
The shifting majorities of the sexes in the workplace.
There’s a widespread attack on insecure MongoDB installs that has resulted in over 28,000 databases being held ransom. This post explains the hack, how to protect yourself and what can we learn from it.
Building a log-based architecture that’s fast, efficient, and resilient on the new stream data structure in Redis.
Complex queries are hard to read. Literate SQL is about writing queries that are as easy to read as a good novel.
Scaling out operation with read replicas and avoiding the downside of stale reads by observing replication progress.
A key part of running a reliable database as a service is ensuring you have a good plan for disaster recovery. This post introduces WAL-G, the successor to WAL-E for Postgres, a complete rewrite of WAL-E that provides the same functionality plus 4x faster restores on recovery.
In many information systems, we work with arrays of integers. For example, maybe you need to keep track of which records in a database contain a given value. This sort of mapping can be expressed as an array of integers. These arrays can end up taking up a large fraction of your memory or disk … Continue reading…
I’ve added a faceted search engine to this blog, powered by PostgreSQL. It supports regular text search (proper search, not just SQL“like” queries), filter by tag, filter by date, filter …
NoSQL databases were the future. MongoDB was the database for “modern” web engineers and used by countless startups. What happened?