The latest blog posts to take off on Hacker News. 95 posts.
With the problem of Image Classification is more or less solved by Deep learning, Text Classification is the next new developing theme in deep learning. For those who don’t know, Text classification is a common task in natural language processing, which transforms a sequence of text of indefinite length into a…
Recently I was working on a project where I have to cluster all the words which have a similar name. For a novice it looks a pretty simple job of using some Fuzzy string matching tools and get this done. However in reality this was a challenge because of multiple reasons starting from pre-processing of the data to…
Household income determines health spending, mostly via higher intensity, not prices. US behaviors are consistent with well-measured income in multiple dimensions.
In this post, we model and explore the two-phase commit protocol using TLA+. The two-phase commit protocol is practical and is used in man...
Ok, enough bickering and fighting.
On December 21, 2018, the United States House of Representatives voted to enact H.R. 4174, the Foundations for Evidence-Based Policymaking Act of 2017, in a historic win for open government in the …
In the summer of 2006, I was lucky enough to be an intern at Google. At the time, Google had an internal tool called gsearch that acted as if it ran grep over all the files in the Google source tree and printed the results. Of course, that implementation would be fairly slow, so what gsearch actually did was talk to a…
As a researcher/engineer (really, an algorithm developer) in the general area of audio and speech processing, I've always run into the same difficulty at all the companies I've worked at: How to quickly prototype and develop algorithms, and subsequently turn them into efficient code that can be deployed to customers.
A handy rule of thumb for creating a confidence interval for the probability of an event you haven't seen happen yet.
An exploration of NFL play-by-play statistics with Bokeh and Pandas.
When I got started using the Datomic database, I remember someone describing it to me as 'Event Sourcing without the hassle'. Having built Event Sourcing systems both with and without Datomic, I think this is very well put, although it might not be obvious, especially if you don't have much experience with Datomic.
Update: I wrote a follow-up article about adding Loop Counters and Row Counts to SQL Plan FlameGraphs. I also show some old examples of “Bookshelf Graph” visualizations there. Check it out after reading this post first. Introduction Brendan Gregg invented and popularized a way to profile & visualize program response…
Discussions: Hacker News (195 points, 51 comments), Reddit r/Python (140 points, 18 comments) If you’re planning to learn data analysis, machine learning, or data science tools in python, you’re most likely going to be using the wonderful pandas library. Pandas is an open source library for data manipulation and…
Building “Test Your Culture” by taking the world’s largest encyclopedia… and combining it with data science
Parallel sequential scan is the first parallel access method in PostgreSQL and is introduced in version 9.6. The committer of this featu...
In my last post, I wrote about my adventure of requesting metadata for both phone calls and emails from the City of Chicago Office of the Mayor. The work there - and its associated frustration - sent me down a path of sending requests throughout the US to both learn whether these sorts of problems are systemic…
Hitting the limit for maximum allowed connections is a common operational problem in Postgres. Here we look at a few techniques for managing connections and making efficient use of those that are available.
I made a New Year's resolution: every plot I make during 2018 will contain uncertainty estimates. Nine months in and I have learned a lot, so I put together a summary of some of the most useful methods.
Datasette (previously) is my open source tool for exploring and publishing structured data. There are a lot of ideas embedded in Datasette. I realized that I haven’t put many of …
Facebook and Google and other advertising businesses are data factories, and regulation will be most effective if it lets users look inside
Decision trees are the fundamental building block of gradient boosting machines and Random Forests(tm), probably the two most popular machine learning models for structured data. Visualizing decision trees is a tremendous aid when learning how these models work and when interpreting models. Unfortunately, current…
The design and implementation of modern column-oriented database systems Abadi et al., Foundations and trends in databases, 2012 I came here by following the references in the Smoke paper we looked…
Postgres 11 is almost here, in fact the latest beta shipped today, and it features a lot of exciting improvements. If you want to get the full list of features it is definitely worth checking out the release notes, but for those who don’t read the release notes I put together a run down of some what I consider the…
(Spanner vs. Calvin, Part 2) [TL;DR I wrote a post in 2017 that discussed Spanner vs. Calvin that focused on performance differences. Th...
I wanted to see how daily patterns emerge at the individual level and how a person’s entire day plays out. So I simulated 1,000 of them.
