91
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 165 posts.
92
Should you ditch Spark for DuckDB or Polars?
There’s been a lot of excitement lately about single-machine compute engines like DuckDB and Polars. With the recent release of pure Python Notebooks in Microsoft Fabric, the excitement about these lightweight native engines has risen to a new high. Out with Spark and in with the new and cool animal-themed engines— is…
93
94
Let's talk about joins
An review of horizontal and vertical joins with examples from education research.
95
131M American Buildings
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
96
97
AI for Data Journalism: demonstrating what we can do with this stuff
I gave a talk last month at the Story Discovery at Scale data journalism conference hosted at Stanford by Big Local News. My brief was to go deep into the …
98
What Shapes Do Matrix Multiplications Like?
Divining order from the chaos
99
Building data infrastructure that will last
As a consultant, I have been called in to review and, in many cases, replace dozens of half-finished, abandoned, and sometimes forgotten data infrastructure projects.
100
Implementing system-versioned tables in Postgres
There's no official support for system-versioned tables in Postgres yet, and the temporal extensions aren't supported on Azure/AWS/GCP. Let's implement it ourselves with 3 triggers and an index.
101
Hacking on PostgreSQL is hard
Hacking on PostgreSQL is really hard. I think a lot of people would agree with this statement, not all for the same reasons. Some might poin...
102
103
1,600 days of a failed hobby data science project
I dedicated over 1,600 days to a data science project, only to see it fail. In this article, I share a new checklist I developed to help prevent similar setbacks in the future.
104
PostgreSQL High Availability Solutions – Part 1: Jepsen Test and Patroni
Note: code used in this article can be found on the Github repo jepsen-postgres-ha .
105
Bcachefs, an Introduction/Exploration
Introduction & background information NOTE: This content is from an internal talk I gave, thus the reason it may read like a presentation So what is …
106
How (and why) to run SQLite in production
This is my personal site, where I write about Ruby, programming, and any of my varied fascinations.
107
Postgres as a Search Engine
Build a retrieval system with semantic, full-text, and fuzzy search in Postgres to be used as a backbone in RAG pipelines.
108
Foursquare Open Source Places: A new foundational dataset
I did not expect this! [...] we are announcing today the general availability of a foundational open data set, Foursquare Open Source Places ("FSQ OS Places"). This base layer of …
109
SQL, Homomorphisms and Constraint Satisfaction Problems
Database queries are a pretty surprisingly powerful tool that can solve seemingly intractable problems.
110
Making database systems usable
C. J. Date's Sigmod 1983 keynote, "Database Usability", was prescient. Usability is the most important thing to the customers. They care les...
111
Lessons Learned from Scaling to Multi-Terabyte Datasets
This post is meant to guide you through some of the lessons I’ve learned while working with multi-terabyte datasets. The lessons shared are focused on what someone may face as the size of the…
112
Fruits (2020)
Fruits - Graph All The Things, a math blog by Martin Loncaric
113
Global EV Charging Points with Open Charge Map
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
114
The Birth of Parquet
15 years ago (2007-2011) I was at Yahoo! working with Map/Reduce and Apache Pig, which was the better Map/Reduce at the time. The Dremel paper just came out and, as everything I worked with seemed to be inspired from Google papers, I read it. I could see it applying to what we were doing at Yahoo! and this was to…
115
116
Knowledge Graphs in RAG: Hype vs. Ragas Analysis
A deep dive on Microsoft's GraphRAG paper found questionable metrics with vaguely defined lift, so I analyzed knowledge graphs in RAG overall using Neo4j vs FAISS
117
Trade-offs between Different CRDTs
🔰If CRDTs are new to you, here are some introductory links. - An interactive intro to CRDTs - An introduction to state-based CRDTs - CRDTs for non-academics - CRDT: The Hard Parts - Readings in CRDTs - crdt.tech What are the trade-offs between different kinds of CRDTs (Conflict-free Replicated
118
Sculpting the moon in R: Subdivision surfaces and displacement mapping
We like the moon The moon is very useful, everyone Everybody like the moon Because it light up the sky at night and it lovely and it makes the tides go and we like it But not as much as cheese We really like cheese
119
Putting a full power search engine in Ecto
My diatribe about adding support for ParadeDB to Ecto
120
Nutrient levels in retail grocery stores
A counterintuitive conclusion