1051
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 2,337 posts.
1052
Postgres data stored in Parquet on S3: LTAP architecture explained
Almost every traditional database keeps its write-ahead log and data files on one machine's disk, which is the root cause of data loss risk, expensive read replicas and high-availability clones, and analytics queries that drag down transactions.
1053
SQL Tricks
Listicles like these do work – not only do they attract attention, if the content is also valuable (and in this case it is, trust me), the article format can be extremely entertaining. This a…
1054
How we achieved write speeds of 1.4M rows per second
The detailed story of how to build a vector-based storage model from scratch.
1055
Feather: A Fast On-Disk Format for Data Frames for R and Python
Introducing Feather: a fast binary file format for data frames that works in both R and Python, built on Apache Arrow.
1056
37signals: A/B testing part 3: Finalé
I was an A/B test skeptic. Maybe we don’t want to be second-guessed. Maybe we don’t want to cater to the lowest common denominator. Designers are taught—explicitly and implicitly—to follow certain visual rules and the final design will work great. The whole A/B testing concept probably …
1057
Hard Drive Stats for Q2 2016
We take a close look at the hard drive failure rates from 68,813 hard drives that are used in the Backblaze data center.
1058
OpenTelemetry at Scale: Using Kafka to handle bursty traffic
Learn how to use Apache Kafka as a buffer between OpenTelemetry Collectors for a scalable multi-collector pipeline, with YAML configuration examples.
1059
CNCF's Cortex v1.0: scalable, fast Prometheus implementation
Grafana is an open source visualization and dashboarding technology that allows you to explore your metrics, logs, and traces no matter where the data is.
1060
1061
Build real-time knowledge graph for documents with LLM
Build a Neo4j knowledge graph from Markdown documents with an LLM. CocoIndex updates the extracted relationships as documents change.
1062
1063
PostgreSQL views and materialized views and how they influenced TimescaleDB
Continuous aggregates are one of Timescale's most popular features, but you can’t fully understand them without some context on Postgres views and materialized.
1064
Postgres isn't slow, your storage is
A recap of Sai Srirampur's POSETTE 2026 talk on storage performance in Postgres, with a local NVMe vs. EBS benchmark and the production setup behind it.
1065
Time Series, the new shiny?
If you’ve been fortunate enough to have been studying penguins at McMurdo these last few seasons you may have missed the attention “Time Series” has been getting in the Big Data space. Then again, maybe not. Because if you were doing scientific work whilst snug in your North Face you may have been t
1066
Building CockroachDB on top of RocksDB
CockroachDB uses RocksDB for its storage engine because of RocksDB's rich feature set, which is necessary for a complex product like a distributed SQL database.
1067
Tendermint 0.10.2
Tendermint is a server and a protocol for building linearizable (or sequentially consistent) byzantine fault-tolerant applications. Tendermint validators accept transactions from clients over HTTP, and replicate them to the other validators in the cluster, forming a totally ordered sequence of transactions. Each…
1068
Facebook's Top Open Data Problems
At Meta, research permeates everything we do. We believe the most interesting research questions are derived from real world problems.
1069
Python Pandas: Tricks and Features
Lesser-known but idiomatic pandas features for those already comfortable with Pandas' basic functionality and concepts.
1070
How We Proved the Eth2 Deposit Contract Is Free of Runtime Errors
Consensys is a blockchain software company building the tools, infrastructure and research that power the new Era of Decentralized Finance, for a more secure and open economy
1071
Index bloat reduced in PostgreSQL v14
This article describes how PostgreSQL v14 reduces B-tree index bloat with "bottom-up index tuple deletion". A test case shows the difference.
1072
1073
The Horrors of Upgrading Etcd Beneath Kubernetes
We dig into the challenges of in-place etcd upgrades beneath autonomous Kubernetes clusters running in air-gapped environments.
1074
.NET Core Image Processing
Image processing, and in particular image resizing, is a common requirement for web applications. As such, I wanted to paint a panorama of the options that exist for .NET Core to process images. For each option, I’ll give a code sample for image resizing, and I’ll outline interesting features. I’ll conclude with a…
1075
CRDT Benchmarks
Open-source TypeScript library for collaborative JSON editing powered by CRDTs.
1076
1077
Storage engine design
In time-series databases the querying pattern differs from the write pattern. We usually write data in time order by updating many series every second. But querying is a different story. We usually want to read only a handful of series leaving most of the data behind. Here lies the biggest problem, which is we don’t…
1078
Postgres Data Flow
What happens when you query Postgres? Data can actually come from many different places like the application cache, buffer cache, and even down into the physical disk cache. This post surveys the data storage and flow of Postgres data.
1079
Using a disk-based Redis clone to reduce AWS S3 bill
Using SSDB to solve the Redis RAM limitation.
1080
Call of Duty: Warzone Caldera Data Set for Academic Use
Access Call of Duty: Warzone Caldera OpenUSD Data Set