601
Push-based outbox pattern with Postgres logical replication
There are only a few patterns that I’m comfortable saying: “You should use it always if you want to build mature system”. One of them is the Outbox Pattern. Why? Because…
New
The latest blog posts to take off on Hacker News. 1,888 posts.
602
Maintaining sufficient free space with ZFS
TLDR: ZFS free-space reporting is a lagging indicator. Background I use Proxmox VM server backed by a ZFS array of hard drives for various personal infrastructure. I also have security cameras that upload motion-triggered videos to my server (via FTP!). Problem Description I would like to use 90% of my available space…
603
Modeling Starlink Capacity
Prior attempts to calculate Starlink’s potential for service have been theoretical. We simulate based on the actual constellation.
604
SAT score distributions in Michigan
The state of Michigan required all public HS seniors to take the SAT last year (~91k out of ~107k total seniors in the state). This generate...
605
Partitioning in Postgres, 2022 Edition
Tracking improvements to Postgres partitioning since 2017. In just five years it’s gone from a non-feature to a very good one.
606
Analysis of Apple Watch running data
Analysis of Apple running measurements and statistics
607
608
609
TIL–Python has a built-in persistent key-value store (2018)
Simplex Sigillum Veri
610
Dabbling with Dagster vs. Airflow
Dissecting the Octopus
611
A sequel to SQL? An intro to Malloy
Homepage of Carlin Eng
612
Extending SQLite with Rust
SQLite has a powerful extension mechanism: loadable extensions. Being an in-process database, SQLite has other extensions mechanisms like application-defined functions (UDF for short). But UDFs have …
613
A Short Chronology of Deep Learning for Tabular Data
Occasionally, I share research papers proposing new deep learning approaches for tabular data on social media, which is typically an excellent discussion…
614
Roaring bitmaps: what they are and how they work
A detailed primer on Roaring bitmaps explaining what they are, how they're different from traditional bitmaps, and how they work internally.
615
Notes on the SQLite DuckDB Paper
SQLite: Past, Present, and Future is a newly published paper authored by Kevin P. Gaffney, Martin Prammer and Jignesh M. Patel from the University of Wisconsin-Madison and D. Richard Hipp, …
616
Understanding Google’s File System (2020)
These paper reviews can be delivered weekly to your inbox , or you can subscribe to the Atom feed . As always, feel free to reach out on Twitter with feedback or suggestions!
617
A Graph-Based Firebase
In A Database in the Browser , I wrote that the schleps we face as UI engineers are actually database problems in disguise [1] . This begged the question: would a database-looking solution solve them?
618
A Critique of SQL, 40 Years Later
Homepage of Carlin Eng
619
What's the big deal about embedded key-value databases?
What's the big deal about key-value databases like FoundationDB and RocksDB?
620
Parsing SQL
SQL is a language to handle data in databases. However, it has limitations and you might need to parse SQL to get around them.
621
Turning SQLite into a Distributed Database
mvSQLite is the open-source , SQLite-compatible distributed database. We decoupled SQLite's storage layer onto FoundationDB to offer massive yet "bottomless" scalability, point-in-time reads, and the strictest level of consistency.
622
How to create a blog post title optimizer with GPT-3 and Hacker News data
GPT-3 says the title for this post is very bad.
623
Excel is pretty dang cool
It's like discovering a treasure chest under my kitchen sink
624
Redis explained
A deep technical dive into all things Redis. Covering various Redis topologies, data persistence and process forking.
625
Reddit’s database has two tables (2012)
Steve Huffman talks about Reddit’s approach to data storage in a High Scalability post from 2010. I was surprised to learn that they only have two tables in their database.
626
Let's Talk SkipList
the forgotten data structure (not really)
627
Having, a less understood SQL clause
Understanding how the group by and having clauses work can help us write more efficient SQL. I want to demonstrate this with an example…
628
629
Airflow's Problem
In 2022, data engineers manage forests, not trees
630
The many flavors of hashing
About the many types of hash functions, their use-cases, dos and don'ts, with suggestions for currently accepted algorithms.