1231
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 1,888 posts.
1232
1233
Analyzing 1.1B NYC Taxi and Uber Trips
The New York City Taxi & Limousine Commission has released a staggeringly detailed historical dataset covering over 1.1 billion individual taxi trips in the city from January 2009 through June 2015. …
1234
I’ll Give MongoDB Another Try In Ten Years
A few weeks ago I wrote a small app that fetches JSON documents from app.net’s API and draws a word cloud . At first I wasn’t keeping the content around after generating the images. Later I thought of other things I’d like to do with the documents, so I decided to start storing them.
1235
Lossless Log Aggregation – Reduce Log Volume by 99% Without Dropping Data
Reduce Log Volume by 99% Without Dropping Data
1236
1237
Upsert in SQL
Inserting and updating data with a single query.
1238
PostgreSQL 10: Partitions of partitions
PostgreSQL version 10 brings a much anticipated feature: (Native) Table Partitioning .
1239
Data Manipulation in Clojure Compared to R and Python
I spend a lot of time developing and teaching people about Clojure's open source tools for working with data. Almost everybody who wants to use Clojure…
1240
Fixing Google Map Transit Feed Mistakes in Taiwan
Probably one of the nerdier things I do in my free time is fix and report transit errors on Google Maps. Accurate transit directions and schedules are extremely important if we want more people to take transit more often, especially in scenarios where good transit options exist! I do these bug reports and the…
1241
Analyzing Last.fm Listening History
Last.fm is a web site that tracks your music listening history across devices (computer, phone, iPod, etc) and services (Spotify, iTunes, Google Play, etc). I’ve been using Last.fm for nearly 10 years now, and my tracked listening history goes back even further when you consider all my pre-existing iTunes play counts…
1242
Recommending music on Spotify with deep learning
An overview of what I've been doing as part of my internship at Spotify in NYC this summer: using convolutional neural networks for audio-based music recommendation.
1243
Call me maybe: final thoughts
Previously in Jepsen , we discussed Riak . Now we’ll review and integrate our findings.
1244
Anonymized data is rarely anonymous
Justin Sherman for Wired points out the farce that is anonymized data: Data on hundreds of millions of Americans’ races, genders, ethnicities, religions, sexual orientations, political beliefs, int…
1245
DELETEs Are Difficult
Why are database DELETEs complex and inefficient? Uncover challenges from physical data removal to autovacuum, and find strategies for managing large deletions.
1246
ZFS Gotchas
Things Nobody Told You About ZFS Yes, it's back. You may also notice it is now hosted on my Blogger page - just don't have time to deal w...
1247
1248
Notes on the Amazon Aurora Paper
This is my notes on the paper: Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases.
1249
1250
1251
Do or undo – there is no VACUUM
What if PostgreSQL didn’t need VACUUM at all? This seems hard to imagine. After all, PostgreSQL uses multi-version concurrency control (MVC...
1252
Optimizing Postgres text search with trigrams
In this post, we’ll implement and optimize a text search system based on Postgres Trigrams
1253
Large language model data pipelines and Common Crawl
This article provides a short introduction to the pipeline used to create the data to train large language models (LLMs) such as LLaMA using Common Crawl (CC).
1254
A Simple Content-Based Recommendation Engine in Python
Let's pretend we need to build a recommendation engine for an eCommerce web site. There are basically two approaches you can take: content-based and collaborative-filtering. We'll look at some...
1255
1256
How to think about creating a dataset for LLM fine-tuning evaluation
I summarise the kinds of evaluations that are needed for a structured data generation task.
1257
Array languages vs. the curse of the spreadsheet
It has been said that the most popular programming language in the world is the spreadsheet (and in particular these days, Excel). There...
1258
1259
1260
