New
The latest blog posts to take off on Hacker News. 86 posts.
Modern SQL: With – Organize Complex Queries
The WITH clause adds named subqueries to SQL queries. WITH is also known as Common Table Expression (CTE).
Happy Little Words – Analyzing the Bob Ross Twitch Chat
In late October the video game streaming site Twitch.tv launched "Twitch Creative" , essentially giving people permission to stream non-video game related creative content on the site. To celebrate the launch they streamed all 403 episodes of The Joy of Painting with Bob Ross in a giant marathon.
Analyzing 1.1B NYC Taxi and Uber Trips
The New York City Taxi & Limousine Commission has released a staggeringly detailed historical dataset covering over 1.1 billion individual taxi trips in the city from January 2009 through June 2015. …
Will ZFS and non-ECC RAM kill your data?
This comes up far too often, so rather than continuing to explain it over and over again, I’m going to try to do a really good job of it once and link to it here.
What MongoDB got right
MongoDB is perhaps the most-widely-mocked piece of software out there right now. While some of the mockery is out-of-date or rooted in misunderstandings, much of it is well-deserved, and it’s difficult to disagree that much of MongoDB’s engineering is incredibly simplistic, inefficient, and immature compared to…
Parallel Sequential Scan is Committed to PostgreSQL 9.6
I previously suggested that we might be able to get parallel sequential scan committed to PostgreSQL 9.5. That did not happen. However, I...
Bayesian ranking of items with up and downvotes or 5 star ratings
Evan Miller wrote a great article How Not To Sort By Average Rating . He shows that simple formulas like score = upvotes - downvotes or score = upvotes / (upvotes + downvotes) don’t work. He proposes to use the lower bound of the Wilson score confidence interval. The formula is complicated and somewhat of a hack…
PostgreSQL on the Command Line
Working efficiently with psql
Document Clustering with Python
ASIAN4D membahas apakah pola Toto4D benar-benar ada, bagaimana cara pemain membacanya, serta insight sederhana yang sering dipakai pemain Indonesia.
How to Analyze Every Reddit Submission and Comment, in Seconds, for Free
With Reddit data in BigQuery, quantifying all the hundreds of millions of Reddit submissions and comments is trivial.
Haunted by Data
This is the text version of a talk I gave on October 1, 2015, at the Strata+Hadoop World conference in New York City. The video version is here (20 mins).
Data first, not code first
Developers hate him! We'll cover some standard tips and tricks here, but we're not really interested in those. We're looking for the One Weird Trick to rule them all. Hopefully each trick we encounter brings us closer to coding Mecca. In the beginning The first video game I ever wrote was called Ninja Wars. Yes, that…
Call Me Maybe: Percona XtraDB Cluster
Percona’s CTO Vadim Tkachenko wrote a response to my Galera Snapshot Isolation post last week. I think Tkachenko may have misunderstood some of my results, and I’d like to clear those up now. I’ve ported the MariaDB tests to Percona XtraDB Cluster, and would like to confirm that using exclusive write locks on all…
Reasons to love SQLAlchemy
I've seen some bashing of ORMs of late, which I think is totally undeserved. I'm a big fan of SQLAlchemy ; I use it in most of my projects, and have contributed code to the library. I'm going to explain 10 good reasons you should love SQLAlchemy too! In fairness, there are a number of good ORMs around, and most of…
The Unreasonable Effectiveness of Random Forests
It’s very common for machine learning practitioners to have favorite algorithms. It’s a bit irrational, since no algorithm strictly…
Call Me Maybe: MariaDB Galera Cluster
Previously, on Jepsen, we saw Chronos fail to run jobs after a network partition . In this post, we’ll see MariaDB Galera Cluster allow transactions to read partially committed state.
Migrating bajillions of database records at Stripe
There are roughly ten thousand bajillion (to the nearest bajillion) merchants registered with Stripe. We recently finished a Very Big Migration of Large Amounts Of Data between several database tables for every single one of them, without losing any of it, without any downtime, mis-reads or mis-writes, whilst running…