Trickest Blog company blog

Getting 10TB of GitHub logs and extracting details of all users and repositories

trickest.com published 2023-06-14

Trickest workflow parses 15TB of GitHub Archive logs and extracts public details for every user and repository back to 2015.

Points

103

HN comments

50

Submissions

1

Cited by

0 commenters

Databases & DataInfrastructure & DevOps2023

Related posts from the same era

I'm all-in on server-side SQLite

Fly.io Blog20221,353 points565 commentscited by 16 details

GPT in 500 Lines of SQL

Explain Extended20241,111 points76 commentscited by 3 details

Datomic is Free

DDatomic Blog20231,099 points368 commentscited by 5 details

My favourite API is a zipfile on the European Central Bank's website

csvbase Blog20231,024 points265 comments details

Suspicious data pattern in recent Venezuelan election

AAndrew Gelman2024903 points470 comments details

Health insurers just published close to a trillion hospital prices

DoltHub Blog2022901 points526 comments details