31
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 86 posts.
32
33
The Universal Data Structure
Whether you write compilers, web services or Django admin panels, you probably use hash maps more than all other data structures combined. In fact, you may not remember the last time you used anything else. This is not a coincidence: any conceivable data structure can be implemented with a hash map; therefore, there…
34
35
Reasons to love SQLAlchemy
I've seen some bashing of ORMs of late, which I think is totally undeserved. I'm a big fan of SQLAlchemy ; I use it in most of my projects, and have contributed code to the library. I'm going to explain 10 good reasons you should love SQLAlchemy too! In fairness, there are a number of good ORMs around, and most of…
36
Why Are Geospatial Databases So Hard to Build?
In 2005, I decided to design my first geospatial database engine. PostGIS was unable to keep up with the modest volumes of sensor data I was working with and while I was new to geospatial data, I had skill designing database engines. It seemed simple enough to get the scale and performance I required. Nothing could…
37
38
What interests Reddit? A network analysis of 84M comments by 200K users
Reddit.com is a popular social media and news aggregation site. Here I analyze over 80 million comments by 200K redditors to discover what topics interest them.
39
40
41
Migrating bajillions of database records at Stripe
There are roughly ten thousand bajillion (to the nearest bajillion) merchants registered with Stripe. We recently finished a Very Big Migration of Large Amounts Of Data between several database tables for every single one of them, without losing any of it, without any downtime, mis-reads or mis-writes, whilst running…
42
43
44
Google’s S2, geometry on the sphere, cells and Hilbert curve
Update - 05 Dec 2017: Google just announced that it will be commited to the development of a new released version of the S2 library, amazing news, repository can be found here. Google's S2 library is a real treasure, not only due to its capabilities for spatial indexing but also because it is a library that
45
Modern SQL: With – Organize Complex Queries
The WITH clause adds named subqueries to SQL queries. WITH is also known as Common Table Expression (CTE).
46
Bayesian ranking of items with up and downvotes or 5 star ratings
Evan Miller wrote a great article How Not To Sort By Average Rating . He shows that simple formulas like score = upvotes - downvotes or score = upvotes / (upvotes + downvotes) don’t work. He proposes to use the lower bound of the Wilson score confidence interval. The formula is complicated and somewhat of a hack…
47
84% of a single-threaded 1KB write in Redis is spent in the kernel
Alex Clemmer is a computer programmer. Other programmers love Alex, excitedly describing him as "employed here" and "the boss's son".
48
49
50
Haunted by Data
This is the text version of a talk I gave on October 1, 2015, at the Strata+Hadoop World conference in New York City. The video version is here (20 mins).
51
What MongoDB got right
MongoDB is perhaps the most-widely-mocked piece of software out there right now. While some of the mockery is out-of-date or rooted in misunderstandings, much of it is well-deserved, and it’s difficult to disagree that much of MongoDB’s engineering is incredibly simplistic, inefficient, and immature compared to…
52
DataLake
A Data Lake is a store that hold raw data as a source for data scientists to explore ways to gain information. It should not be accessed by end-users or used for system integration.
53
54
55
Analyzing 1.1B NYC Taxi and Uber Trips
The New York City Taxi & Limousine Commission has released a staggeringly detailed historical dataset covering over 1.1 billion individual taxi trips in the city from January 2009 through June 2015. …
56
How to Analyze Every Reddit Submission and Comment, in Seconds, for Free
With Reddit data in BigQuery, quantifying all the hundreds of millions of Reddit submissions and comments is trivial.
57
Some analysis of the 1M most popular sites on the web
Over the last couple of weeks I’ve run some analysis on the one million most popular sites on the web.
58
Catena: A time series storage engine written in Go
First, I suggest reading Baron’s “Time-Series Database Requirements” blog post to get some more context for this post. I read that and, as I usually do, had my mind set on low-level thoughts. I wrote the following comment:
59
Happy Little Words – Analyzing the Bob Ross Twitch Chat
In late October the video game streaming site Twitch.tv launched "Twitch Creative" , essentially giving people permission to stream non-video game related creative content on the site. To celebrate the launch they streamed all 403 episodes of The Joy of Painting with Bob Ross in a giant marathon.
60