2221
OpenZFS – dRAID, Finally
Learn about dRAID in OpenZFS 2.1. Explore its data protection capabilities, distributed hot spares, faster resilvering, and enhanced performance in this article.
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 4,224 posts.
2222
Eventual Consistency isn’t for Streaming
Understand why eventual consistency isn't suitable for streaming systems & the systematic errors it can cause with Materialize's insights.
2223
About ZFS Performance
You likely know I like the ZFS filesystem a lot. ZFS stores the files in B-trees in a similar fashion as InnoDB stores data.
2224
Pipelining in psql (PostgreSQL 18)
the psql client version 18 comes with pipelining, which can speed up client-server communication. In this post, let's see how it works and how much can be g...
2225
Unfinished Business with Postgres
7 years ago I left Heroku. Heroku has had a lot of discussion over the past weeks about its demise, whether it was a success or failure, and the current state. Much of this was prompted by the recent, and on-going security incident, but as others have pointed out the product has been frozen in time for some years now…
2226
Using Python and OCaml in the same Jupyter notebook
The cover image is based on Jupiter family by NASA/JPL.
2227
Understanding Postgres Performance
Update theres a more recent post that expands further on where to start optimizing specific queries, and of course if you want to dig into optimizing your infrastructure High Performance PostgreSQL is still a great read For many application developers their database is a black box. Data goes in, comes back out and in…
2228
2229
2230
We sped up time series by 20-30x
This is a follow up post that dives into how we managed to achieve such huge performance gains for time series (among other things), and why it was so hard in the first place.
2231
Pgsql: Create an infrastructure for parallel computation in PostgreSQL
Create an infrastructure for parallel computation in PostgreSQL. This does four basic things. First, it provides convenience routines to coordinate …
2232
An Introduction to Conflict-Free Replicated Data Types
This is an interactive tutorial series about Conflict-Free Replicated Data Types, or CRDTs for short. Their purpose is to allow seamless replication of data on different nodes in a distributed system. Merging is by construction always possible, without any conflicts. This series assumes no knowledge about CRDTs, but…
2233
Quicker serverless Postgres connections
Neon’s serverless driver redirects the PostgreSQL TCP wire protocol over WebSockets. This makes ordinary, fully-functional PostgreSQL connections accessible from new environments — including serverless platforms like Cloudflare Workers and Vercel Edge Functions. A key feature of these environments is that state is not…
2234
Amazon S3 Transfer Acceleration
Several AWS teams are focused on simplifying and accelerating the process of moving on-premises data to the cloud. We started out with the very basic PUT operation and multipart upload in the early days. Along the way we gave you the ability to send us a disk, and made that process even easier by launching […]
2235
How I Write SQL, Part 1: Naming Conventions (2014)
"There are only two hard problems in Computer Science: cache invalidation and naming things."
2236
RethinkDB 1.6 is out: regex matching, new array operations, random sampling
This release includes regex string matching, fourteen new array operations, support for random sampling, error handling improvements, support for authentication keys, and over 65 bug fixes, features, and enhancements.
2237
How should you build a high-performance column store for the 2020s?
Though most relational databases (like MySQL) are "row oriented", in that they keep rows stored together... experience has taught us that pivoting the data arrow so that columns, and not rows, are stored together can be beneficial. This is an old observation that most experienced programmers know about as the…
2238
Deploying PostgreSQL Clusters Using StatefulSets
Editor’s note: Today’s guest post is by Jeff McCormick, a developer at Crunchy Data, showing how to build a PostgreSQL cluster using the new Kubernetes StatefulSet feature. In an earlier post, I described how to deploy a PostgreSQL cluster using Helm, a Kubernetes package manager. The following example provides the…
2239
2240
The Unbundling of Airflow
Star us on GitHub In each subdomain of software, we have seen the same story over and over again. Products start small, in time, add adjacent verticals and functionality to their offerings and end up becoming a platform. Once these platforms become big enough, people start to figure out how
2241
2242
WordPress to support SQLite back end
A few months ago, a proposal was published to make WordPress officially support SQLite. After the proposal received a lot of positive feedback from the community, we started working on an implement…
2243
Phases of Netflix’s real-time data infrastructure
The blog post will share the four phases of Real-time Data Infrastructure’s iterative journey in Netflix (2015-2021). For each phase, we will go over the evolving business motivations, the team’s unique challenges, the strategy bets, and the use case patterns we discovered along the way.
2244
Postgres as a Graph Database: (Ab)Using PgRouting
Learn how to use pgRouting as a lightweight graph database solution in Postgres.
2245
2246
2247
How we built ngrok's data platform
At ngrok, we manage a ~100TiB, 500+ table data lake, managed by a very small team. Here's a look at how we built it and what unique challenges we solved.
2248
Postgres performance analysis resulted in a 10x improvement in CPU use
Heap is the only digital insights platform that shows everything users do on your site, revealing the "unknown unknowns" that stay invisible with other tools. Learn about how we improved our performance and saved millions!
2249
How to implement Q&A against your docs with GPT3 embeddings and Datasette
If you’ve spent any time with GPT-3 or ChatGPT, you’ve likely thought about how useful it would be if you could point them at a specific, current collection of text …
2250
