On 2026-05-30, the Mastodon server I run started getting a flood of implausible signup requests. We think these are likely bots because our server rules and signup form clearly tell users to talk about their interest in gay leather. Most people sign up with something like “42 yo levis enthusiast into piss” or “Hi! I’m…
Visit aphyr.com → RSS feedOn Hacker News May 2013 – Apr 2026
Systems & Performance 24Databases & Data 23Software Engineering 13Society, Policy & Culture 8Career & Management 6Programming Languages 5
Please note: our followup analysis of 3.4.0-rc3 revealed additional faults in MongoDB’s replication algorithms which could lead to the loss of acknowledged documents–even with Majority Write Concern, journaling, and fsynced writes.
This post covers Elasticsearch 1.1.0. In the months since its publication, Elasticsearch has added a comprehensive overview of correctness issues and their progress towards fixing some of these bugs.
Previously, on Jepsen, we reviewed Elasticsearch’s progress in addressing data-loss bugs during network partitions. Today, we’ll see Aerospike 3.5.4, an “ACID database”, react violently to a basic partition.
In this Jepsen report, we’ll verify RethinkDB’s support for linearizable operations using majority reads and writes, and explore assorted read and write anomalies when consistency levels are relaxed. This work was funded by RethinkDB, and conducted in accordance with the Jepsen ethics policy .
In the last Jepsen post, we found that RethinkDB could lose data when a network partition occurred during cluster reconfiguration. In this analysis, we’ll show that although VoltDB 6.3 claims strict serializability, internal optimizations and bugs lead to stale reads, dirty reads, and even lost updates; fixes are now…
Previously, on Jepsen, we saw Chronos fail to run jobs after a network partition . In this post, we’ll see MariaDB Galera Cluster allow transactions to read partially committed state.
Previously, on Jepsen , we demonstrated stale and dirty reads in MongoDB . In this post, we return to Elasticsearch, which loses data when the network fails, nodes pause, or processes crash.
Cassandra is a Dynamo system; like Riak, it divides a hash ring into a several chunks, and keeps N replicas of each chunk on different nodes. It uses tunable quorums, hinted handoff, and active anti-entropy to keep replicas up to date. Unlike the Dynamo paper and some of its peers, Cassandra eschews vector clocks in…
Chronos is a distributed task scheduler (cf. cron) for the Mesos cluster management system. In this edition of Jepsen , we’ll see how simple network interruptions can permanently disrupt a Chronos+Mesos cluster
RabbitMQ is a distributed message queue, and is probably the most popular open-source implementation of the AMQP messaging protocol. It supports a wealth of durability, routing, and fanout strategies, and combines excellent documentation with well-designed protocol extensions. I’d like to set all these wonderful…
A few weeks ago I criticized a proposal by Antirez for a hypothetical linearizable system built on top of Redis WAIT and a strong coordinator. I showed that the coordinator he suggested was physically impossible to build, and that anybody who tried to actually implement that design would run into serious problems. I…
In the previous post, we discovered the potential for data loss in RabbitMQ clusters . In this oft-requested installation of the Jepsen series, we’ll look at etcd : a new contender in the CP coordination service arena. We’ll also discuss Consul ’s findings with Jepsen.
In the previous Jepsen analysis of RethinkDB , we tested single-document reads, writes, and conditional writes, under network partitions and process pauses. RethinkDB did not exhibit any nonlinearizable histories in those tests. However, testing with more aggressive failure modes, on both 2.1.5 and 2.2.3, has…
In a recent blog post , antirez detailed a new operation in Redis: WAIT . WAIT is proposed as an enhancement to Redis’ replication protocol to reduce the window of data loss in replicated Redis systems; clients can block awaiting acknowledgement of a write to a given number of nodes (or time out if the given threshold…
Previously in Jepsen , we discussed Riak . Now we’ll review and integrate our findings.
Some folks have asked whether Cassandra or Riak in last-write-wins mode are monotonically consistent, or whether they can guarantee read-your-writes, and so on. This is a fascinating question, and leads to all sorts of interesting properties about clocks and causality.
In the last Jepsen analysis, we saw that RethinkDB 2.2.3 could encounter spectacular failure modes due to cluster reconfiguration during a partition. In this analysis, we’ll talk about Crate, and find out just how many versions a row’s version identifies.
Percona’s CTO Vadim Tkachenko wrote a response to my Galera Snapshot Isolation post last week. I think Tkachenko may have misunderstood some of my results, and I’d like to clear those up now. I’ve ported the MariaDB tests to Percona XtraDB Cluster, and would like to confirm that using exclusive write locks on all…
Previously in Jepsen , we discussed Redis . In this post, we’ll see MongoDB drop a phenomenal amount of data. See also: followup analyses of 2.6.7 and 3.4.0-rc3 .
This article is part of Jepsen , a series on network partitions. We’re going to learn about distributed consensus, discuss the CAP theorem’s implications, and demonstrate how different databases behave under partition.
Previously on Jepsen , we introduced the problem of network partitions . Here, we demonstrate that a few transactions which “fail” during the start of a partition may have actually succeeded.
Previously on Jepsen , we explored two-phase commit in Postgres . In this post, we demonstrate Redis losing 56% of writes during a partition.
Previously in Jepsen , we discussed Redis . In this post, we’ll see MongoDB drop a phenomenal amount of data. See also: followup analyses of 2.6.7 and 3.4.0-rc3 .