301
Fundamentals of Object Storage in Elixir
I did a livestream where I talked about Object Storage. The how and why. The Bad Old Days. And also the neat and interesting stuff just beyond the basics. I figured I'd cover that in text as well.
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 350 posts.
302
303
Postgres vs. Pinecone
We respond to Pinecone's recent blog post comparing Postgres and Pinecone. We show that Postgres can outperform Pinecone in the same benchmarks Pinecone covered in their article.
304
Optimizers: The Low-Key MVP
The query optimizer is an important part of any analytical database system as it provides considerable performance improvements compared to hand-optimized queries, even as the state of your data changes.
305
Ask questions of SQLite databases and CSV/JSON files in your terminal
I built a new plugin for my sqlite-utils CLI tool that lets you ask human-language questions directly of SQLite databases and CSV/JSON files on your computer. It’s called sqlite-utils-ask. Here’s …
306
Prolly Trees
A deep dive into the Probabilistic B-tree or Prolly Tree data structure including examples. Prolly Trees are used to create version controlled databases.
307
Streaming joins are hard
Why are streaming joins so tricky? Discover the hidden complexities of real-time data, and learn how to tame the infinite stream!
308
New discovery reveals how diatoms capture CO2 so effectively
Tiny diatoms in the ocean are masters at capturing carbon dioxide (CO2) from the environment. They fix up to 20 percent of the Earth’s CO2. A research team at the University of Basel has now discovered a protein shell in these algae that is necessary for efficient CO2 fixation. This groundbreaking discovery can…
309
Quickwit 0.8: Indexing and Search at Petabyte Scale
Quickwit 0.8 brings indexing and search at an unprecedented scale, indexing at 1PB/day and searching through 50 PB is a reality.
310
Valkey achieved one million RPS 6 months after forking from Redis
Maximize the performance of your hardware with memory access amortization
311
Your database skills are not 'good to have' (2023)
And a hateful ode to Object-Relational Mappers
312
Building a streaming SQL engine with Arrow and DataFusion
Arroyo 0.10 has an entirely new SQL engine built with Apache Arrow and DataFusion. It's much faster, smaller, and easier to run. Read on for why and how we're making this change.
313
HBase Deprecation at Pinterest
Alberto Ordonez Pereira | Senior Staff Software Engineer; Lianghong Xu | Senior Manager, Engineering;
314
315
DuckDB Isn't Just Fast
A whistlestop tour of the cool bits of DuckDB
316
Postgres 17: Streaming I/O for sequential scans and ANALYZE
We discuss streaming and vectored I/O in Postgres 17. This is an important step towards implementing asynchronous I/O in Postgres. In Postgres 17 we do not yet have asynchronous I/O, but we do see a performance benefit from what is essentially a refactoring work around the internal APIs.
317
10 Years of Hacker News "Ask HN: Who Is Hiring" Trends
The "Ask HN: Who is hiring" monthly post on Hacker News, available on the first day of each month, provides a platform for users to share tech job listings for free. Analyzing 10 years of these posts with a Python script reveals significant shifts in hiring trends. This insight into the evolving tech job market can…
318
Proof of the Singular Value Decomposition
The existence claim for the singular value decomposition (SVD) is quite strong: “Every matrix is diagonal, provided one uses the proper bases for the domain and range spaces” (Trefethen & Bau III, 1997) . MIT professor Gilbert Strang has a wonderful lecture on the SVD, and he includes an existence proof for the SVD…
319
90x Faster Than Pgvector – Lantern's HNSW Index Creation Time
Index creation time is a critical database metric. Learn more about how Lantern enables 90x faster index creation times than pgvector, and how Lantern compares to Pinecone.
320
The "it" in AI models is the dataset
I’ve been at OpenAI for almost a year now. In that time, I’ve trained a lot of generative models. More than anyone really has any right to train. As I’ve spent these hours observing the effects of tweaking various model configurations and h
321
PostgreSQL is eating the database world
PostgreSQL isn’t just a simple relational database; it’s a data management framework with the potential to engulf the entire database realm. The trend of “Using Postgres for Everything” is no longer limited to a few elite teams but is becoming a mainstream best practice.
322
Django SQLite Production Config
Optimize your Django SQLite configuration for production.
323
Versioned finite-state machines with Postgres (2019)
Inspired by Felix Geisendorfer blog post I implemented a database FSM (Finite-State Machine) with Postgresql. I brought some improvements to Felix’s implementation but before reading the following I recommend you to read carefully the original post.
324
Building Databases over a Weekend
In this post we take you on a walkthrough on how you can use DataFusion to build your own developer experience for databases.
325
CSVs Are Kinda Bad. DSVs Are Kinda Good
I would simply use delimiter characters to delimit data.
326
Self-hosting a high-availability Postgres cluster on Kubernetes
Ryan Schachte — infrastructure, networking, and distributed systems
327
Italian music through the lens of complex networks
Last year I was talking with a non-Italian, trying to convey to them how nearly the entirety of contemporary Italian music rests on the shoulders of Gianni Maroccolo — and the parts that don’t, should. In an attempt to find a way out of that conversation, they casually asked “wouldn’t it be cool to map out who…
328
DataStax just bought our startup Langflow
The past few weeks have been nuts, and we’ve been itching to share some huge news with you all… 🥁 Langflow is joining DataStax!
329
Databases in 2023: A Year in Review
Andy recounts the rise of vector databases to SQL:2023 to MariaDB troubles and the FAA outage in 2023.
330
What Happens When You Put a Database in the Browser?
Explore DuckDB Wasm for fast, in-browser analytical SQL. Learn how to instantiate the JS API, query Parquet files, and export data to CSV with zero latency. | Reading time: 8 min read