Twing Data Blog company blog

Building an open data pipeline in 2024

by Dan Goldin blog.twingdata.com published 2024-04-26

Using Iceberg allows us to pick the optimal "big data" compute environment for the specific requirements we have. There's no need to limit yourself to a single solution.

Points

103

HN comments

32

Submissions

1

Cited by

0 commenters

Databases & DataInfrastructure & DevOps2024

Related posts from the same era

Ingesting PDFs and why Gemini 2.0 changes everything

Sergey's Blog20251,303 points447 comments details

GPT in 500 Lines of SQL

Explain Extended20241,111 points76 commentscited by 3 details

Datomic is Free

DDatomic Blog20231,099 points368 commentscited by 5 details

Postgres IDE in VS Code

Microsoft Tech Community20251,040 points407 comments details

My favourite API is a zipfile on the European Central Bank's website

csvbase Blog20231,024 points265 comments details

The DuckDB Local UI

DuckDB Blog2025926 points188 commentscited by 3 details