A comprehensive guide to CRDTs and their tradeoffs, from counters to sequences. Written in the spirit of the Typeclassopedia, exploring how different CRDTs solve the distributed consensus puzzle.
Now that Joel0 in the TrueNAS community has created a fork of TrueNAS that runs on Arm, I thought I'd give it a spin—on a Raspberry Pi. I currently run an Ampere Arm server in my rack with Linux and ZFS as my primary storage server, and a Raspberry Pi with four SATA SSDs and ZFS as backup replica in my studio. My…
In the previous posts, we've explored the SQLite file format and built a simple SQL parser. It's time to put these pieces together and implement a query…
Yes, you read that right. SQL does treat all NULLs as distinct values. I learnt this a while back while working on Convoy and again on LiteQueue: a Golang a queueing library. Basically, any column wi
Jonathan Arns's personal blog
The adoption of Parquet version 2 is limited due to the lack of support in the ecosystem, which affects its evolution despite its improvements in compression and performance.
I've been working on a complex multi-model database for a few weeks now, and recently I took time to simplify and test out an idea I had on a simple key-value database. I started with the basics: A hash table in memory, a simple append-only log
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
That transfer to John Smith isn't covered by any of the tagging rules. So I select the date, switch to the editor window and paste the date into the date->tag definition:
How to print my voice? How to put music on paper? How to print an executable program? All this boils down to storing digital data on paper. This is also called paper data storage . Arrived in this domain from poetry, I’ve been investigating the area for some time. Here is what I found.
The story behind how LinkedIn created Apache Kafka
From simple object storage to sophisticated table management, builders have always shaped S3's evolution. Andy Warfield discusses why making complex systems simple remains our north star at AWS.
DeepSeek is pushing DuckDB beyond its single-node roots with smallpond, a new, simple approach to distributed compute. But does it solve the scalability challenge—or introduce new trade-offs?
An analysis of Product Hunt, 2014 - 2021
In my previous post I showed how LLMs struggle with coding in q/kdb+; having immediate feedback loops using the q-MCP server does help a…
This post was originally given as a talk for Friendly.rb and Kaigi on Rails. The slides are also available. Between Litestack and the Rails 8 trifecta of Solid Cable, Solid Cache, and Solid Queue, it’s easier than ever to spin up a Rails app that doesn’t need a database service, or a redis service, or a file storage…
Some tricks and techniques for working with graph or tree like data in Postgresql
This article will show you how forming your Pokémon team can be turned into a mathematical optimization problem.
Welcome to my personal blog! I use it to share what I'm currently learning or thinking about, usually on topics related to technology, business, and health.
Databases are global, shared, mutable state. That’s the way it has been since the 1960s, and no amount of NoSQL has changed that. However, most self-respecting developers have got rid of mutable global variables in their code long ago. So why do we tolerate databases as they are?
Word2vec is a pervasive tool for learning word embeddings. Its success, however, is mostly due to particular architecture choices. Transferring these choices to traditional distributional methods makes them competitive with popular word embedding methods.
The oldest recorded transaction was in 3100 BC
The case for less rigor and more nuance.
I analysed 58,687 movie posters from the past century to see which colours dominate, how their use has shifted over time and what each hue signals to audiences.
Lately, there has been a lot of excitement around Durable Execution (DE) engines. The basic idea of DE is to take (potentially long-running) multi-step workflows, such as processing a purchase order …
An in-depth guide to listening to Postgres database changes through the WAL. Covers logical replication, publications, replication slots, and an Elixir implementation.
As programmers we spend a lot of time shuttling data back and forth between different systems and transforming it from one format to another. Sometimes it gets pretty miserable!
Making vector search fast, memory-efficient, and natural in SQLite.
On trying to mash up SQLite with ideas stolen from Accountants, Clojure, Datomic, XTDB, Rama, and Local-first-ers, to satisfy Henderson's Tenth Law. Viz., to make a sufficiently complicated data system containing an ad-hoc, informally-specified, bug-ridden, slow implementation of half of a bitemporal database…
The Marginalia Search index has been partially rewritten to perform much better, using new data structures designed to make better use of modern hardware. This post will cover the new design, and will also touch upon some of the unexpected and unintuitive performance characteristics of NVMe SSDs when it comes to read…
