Data Engineering Central personal blog

650GB of Data (Delta Lake on S3). Polars vs. DuckDB vs. Daft vs. Spark

by Daniel Beach dataengineeringcentral.substack.com published 2025-11-12

cluster fatigue

Points

263

HN comments

107

Submissions

1

Cited by

0 commenters

Databases & DataSystems & Performance2025

Related posts from the same era

ICE using Palantir tool that feeds on Medicaid data

EFF Deeplinks20261,475 points969 comments details

Ingesting PDFs and why Gemini 2.0 changes everything

Sergey's Blog20251,303 points447 comments details

Tracking down the 16-year-old WAL-reset SQLite bug

Tailscale Blog20261,223 points239 commentscited by 3 details

GPT in 500 Lines of SQL

Explain Extended20241,111 points76 commentscited by 3 details

Postgres IDE in VS Code

Microsoft Tech Community20251,040 points407 comments details

PGSimCity - How PostgreSQL Works

Nnikolays.github.io2026930 points90 comments details