271
Dbt Labs acquires SDF Labs
dbt Labs has acquired SDF Labs to enhance analytics engineering solutions. Learn how this partnership strengthens the modern data stack.
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 299 posts.
272
Selective async commits in PostgreSQL – balancing durability and performance
Safely leverage PostgreSQL's asynchronous commit for significant performance gains
273
Data engineering and software engineering are converging
The official JS client for connecting to ClickHouse.
274
I solved a distributed queue problem after 15 years
Learn how queues make horizontal scaling, scheduling, and flow control easier in cloud systems, and how to make them durable and observable.
275
Representing Python notebooks as dataflow graphs
How and why marimo represents notebooks as dataflow graphs
276
OpenTelemetry protocol with Apache Arrow
We are excited to announce the next phase of the OpenTelemetry Protocol with Apache Arrow project (OTel-Arrow). We began this project several years ago with the goal of bridging between OpenTelemetry data and the Apache Arrow ecosystem. Apache Arrow is a framework designed for zero-copy exchange of structured data…
277
278
How NASA Is Using Graph Technology and LLMs to Build a People Knowledge Graph
Missed NASA’s People Graph webinar? Catch the recap and see how graph technology and AI are shaping the future of workforce intelligence.
279
SQLite (with WAL) doesn't do `fsync` on each commit under default settings
SQLite doesn’t do fsync unless specified.
280
The Evaporative Cooling Effect in Social Networks (2010)
http://blog.bumblebeelabs.com/social-software-sundays-2-the-evaporative-cooling-effect/
281
Keeping our free tier sustainable by preventing abuse
A free tier allows people to try a SaaS without paying first, but it opens up the service to bad actors. Here's how we balance preventing bad actors while still supporting legitimate users.
282
ClickHouse vs PostgreSQL UPDATE performance comparison
See how ClickHouse’s update speed stacks up against PostgreSQL on the same data and hardware.
283
MySQL at Uber
At Uber, our MySQL® fleet is the backbone of our data infrastructure, supporting a vast array of operations critical to our platform. Uber operates an extensive MySQL fleet, consisting of over 2,300 independent clusters. Building a control plane to manage this fleet at such a massive scale, while ensuring zero…
284
Replacing EBS and Rethinking Postgres Storage from First Principles
Fluid Storage reimagines block storage like EBS with zero-copy forks, true elasticity, and sync replication. Built for the age of agents.
285
Derivation and Intuition behind Poisson distribution
Made in Notion, the connected workspace where AI agents and teams organize docs, projects and knowledge together.
286
Apache Iceberg now supports geospatial data types natively
Apache Iceberg and Parquet now support native geospatial types, connecting Wherobots, PostGIS, Sedona, Databricks, and Snowflake to one table format.
287
Weighting an average to minimize variance
How would you allocate money to a set of independent assets with to minimize volatility if some assets are more volatile than others?
288
Using the expand and contract pattern for schema changes
In this article, we introduce the expand and contract pattern to help migrate data and clients to a new schema.
289
Extending That XOR Trick to Billions of Rows
Learn how to extend the classic XOR trick to find thousands of missing values using Invertible Bloom Filters
290
SQLite transactions and virtual tables
In my previous post I introduced SQL virtual tables and how to use them in Go—registering modules per connection, defining schemas, and querying external sources as regular tables.
291
Wrong ways to use the databases, when the pendulum swung too far
Disclaimer: This story took place when I was a junior developer for a previous employer ages ago. Some details might be fuzzy, and goes without saying this is entirely from my own POV so take everything here with humor and a grain of salt.
292
Tracing Military Command Chains Through Time and Location
Case Study ⸺ Using novel technology to analyze groundbreaking chain of command research
293
Building agents using streaming SQL queries
Table of Contents Agents Need to Interact With LLMs Agents Should Be Event-Driven Agents Need Context Agents Require Memory When SQL Is Not Enough Parting Thoughts AI Agents have improved in leaps …
294
How/why to sweep async tasks under a Postgres table
I like slim and stupid servers, where each endpoint wraps a very dumb DB query. Dumb queries are fast. Fast queries make websites smooth and snappy. Keep those click/render loops sacred. Sweep complexity under a task table.
295
Using watermarks to coordinate change data capture in Postgres
Explore the challenges of building a change data capture (CDC) pipeline for Postgres. And see how Sequin used a watermark design inspired by Netflix's DBLog.
296
297
298
Writing a storage engine for Postgres: An in-memory table access method (2023)
Writing a storage engine for Postgres: an in-memory Table Access Method
299
AA-Omniscience v1.1: Knowledge and Hallucination Benchmark
Compare AI model performance on AA-Omniscience v1.1: Knowledge and Hallucination Benchmark. A benchmark measuring factual recall and hallucination across various economically relevant domains.