Topics /
The most popular blog posts about databases & data on Hacker News. 299 posts.
XTX Markets is a leading algorithmic trading firm which uses state-of-the-art machine learning technology to produce price forecasts for over 50,000 financial instruments across equities, fixed income, currencies, commodities and crypto. It uses those forecasts to trade on exchanges and alternative trading venues, and…
rqlite is a lightweight, open-source, distributed relational database written in Go, and built on SQLite and Raft. With its origins dating back to 2014, its design has always prioritized reliability, and quality. The robustness of rqlite is also a testament to its disciplined testing strategy: after more than 10 years…
Sharded Postgres by the team behind Vitess
Make your own AI by hand ✍️ exercises
Valkey is not only thriving, but now outperforming Redis 8.0 in real world benchmarks.
Right before Christmas we discovered that our Aurora Postgres instance needed a major version upgrade. This covers how we made the upgrade with zero seconds of downtime.
Exploring LSM tree implementation in PostgreSQL for better write performance.
Historically I have worked on many projects related to high-performance Protobuf, be that on the C++ runtime, on the Rust runtime, or on integrating UPB , the fastest Protobuf runtime, written by my colleague Josh Haberman . I generally don’t post directly about my current job, but my most recent toy-turned-product is…
Never store embeddings in a CSV!
Colossus’s performance comes from its stateful protocol, which plays a pivotal role in the new Rapid Storage offering launched at Google Cloud Next '25.
searchcode.com’s SQLite database is probably one of the largest in the world, at least for a public facing website. It’s actual size is 6.4 TB. Which is probably 6 terabytes bigger than yours. …
Since early 2022, Motion was on CockroachDB. Cockroach has many qualities going for it: effortless horizontal scaling, especially when…
Redis is arguably the most well regarded tech of all. But many times you do not need Redis. Here are 3 examples from my last 3 jobs.
TigerBeetle is an Online Transactional Processing (OLTP) database built for double-entry accounting with a strong emphasis on safety and speed. It builds on the Viewstamped Replication (VR) consensus protocol to offer Strong Serializable consistency. Unlike general-purpose databases, TigerBeetle stores only accounts…
See how we diagnosed and confirmed an AWS Aurora RDS race condition impacting failovers.
The original motivation for the creation of Bloom filters is efficient set membership, using a probabilistic approach to significantly reduce the time and space required to reject items that are not members in a certain set.
The Library Innovation Lab is growing knowledge and community by bringing library principles to technological frontiers.
Introducing the $5 PlanetScale plan.
We benchmarked 11 data formats, including markdown tables, JSON, CSV, YAML, XML, and more, to see which LLMs understand best. We found surprising accuracy and token cost trade-offs.
How Postgres’ v7 UUIDs are made monotonic, and why that’s a great feature.
Distributed coordination makes application complex and brittle. Because all systems eventually build on logs, we can use a shared-log approach to eliminate most coordination. We discuss this conceptually and show how this is practically implemented in the open source project Restate.
Anecdotally, this might be the most requested feature in SQL: Allow some trailing commas.
We need to talk about the data. Crap data. We’re destroying our environment to create and store trillions of blurred images, half-baked videos, rip-off AI ‘songs’, rip-off AI animations, videos and images, emails with mega attachments, never-to-be-watched-again presentations, never-to-be-read-again reports, files and…
See how PostgreSQL 19 speeds up aggregations by aggregating before joins, with a benchmark showing over 5x faster execution.
Intel's latest processor generations are pushing the number of cores in a server to unprecedented levels. For analytical databases like ClickHouse, ultra-high core counts represent a huge opportunity and a complex challenge.
What happens when you give a distributed SQL engine a trillion-row dataset? GizmoEdge crushed the Coiled challenge with a 1,000-worker cluster on Azure.
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
