Top posts
Every blog post that took off on Hacker News, ranked by the votes of its best submission. 4,226 posts.
Using Graph Databases to Stop E-commerce Fraud in Real Time
Discover how to detect and deter e-commerce fraud – before fraudsters inflict significant damage – when you harness the power of graph database technology.
Ember and D3: Building responsive analytics
The Corner is Square's technical blog covering software engineering, APIs, data, open source and more.
How much memory does a graph take?
We need a fast way to query multiple potentially large sets of data on-demand at interactive speeds. Sometimes the easiest solution to a hard problem is to build the right tool for the job.
Optimizing Magic Pocket for cold storage
Dash knows your context, your team, and your work, so your team can stay organized, easily find and share knowledge, and keep projects secure, all from one place.
A Logic Language for Distributed SQL Queries
A comprehensive look at the design of Polar, and how we enabled it to operate over distributed data.
Streaming Messages from Kafka into Redshift in Near Real-Time
Streaming Messages from Kafka into Redshift in near Real-Time Shahid C., Software Engineer Oct 17, 2016 This post is part of a series covering Yelp's real-time streaming data infrastructure. Our...
SQLite transactions and virtual tables
In my previous post I introduced SQL virtual tables and how to use them in Go—registering modules per connection, defining schemas, and querying external sources as regular tables.
Layers in Software: From Data to Value
Then Back in the 2000s, we wrote applications in layers. Presentation layer, client, data transfer, API, business logic, data access, database. We maintained strict separation bet ween these layers…
The Economics of Building Knowledge Bases
During the summers of my high school years in suburban Georgia, my friend and I would fill the time by randomly walking into local establishments asking for odd jobs. It was a great way as a studen…
Type information for faster Python C extensions
Update: The paper version of this post is accepted at PLDI SOAP 2024. Take a look at the preprint (PDF).
Pyleus: An Open-source Framework for Building Storm Topologies in Pure Python
Introducing Pyleus: An Open-source Framework for Building Storm Topologies in Pure Python Patrick L., Software Engineer Oct 15, 2014 Yelp loves Python, and we use it at scale to power...
Why Postgres RDS didn't work for us
Background
The 74,000 numbers of Barclays Bank
The UK faces an epidemic of telephone scams. Fraudsters are constantly calling people up pretending to be their bank. But how can you be sure the number displayed on your screen in genuine? You can't. The telecom system is hopelessly insecure and shouldn't be trusted for anything more complicated than dialling the…
Wrong ways to use the databases, when the pendulum swung too far
Disclaimer: This story took place when I was a junior developer for a previous employer ages ago. Some details might be fuzzy, and goes without saying this is entirely from my own POV so take everything here with humor and a grain of salt.
Replacing Pandas with Polars
I remember those days, oh so long ago, it seems like another lifetime. I haven’t used Pandas in many a year, decades, or whatever. We’ve all been there, done that. Pandas I mean. I would dare say it’s a rite of passage for most data folk. For those using Python, it’s probably one of the […]
Poor schemas, poor cataloguing: why music tagging sucks
One of the things that frustrates me with having a local music library is the tedium of tagging. While there are tools (like beets or MusicBrainz Picard) to make it easier, I feel there are fundamental issues with the design… Continue reading →
Elasticsearch is dead, long live Open Distro for Elasticsearch
In this blog, we describe how we leverage the authentication and authorization support in Open Distro for Elasticsearch to make elasticsearch a...
Building a Database in the 2020s (2022)
Building a database in the 2020s
118M Queries per Second on Neki
We ran a massive, sharded Postgres database at 118.5 million queries per second, with 200k queries per second on each shard across 512 shards.
Getting 10TB of GitHub logs and extracting details of all users and repositories
Trickest workflow parses 15TB of GitHub Archive logs and extracts public details for every user and repository back to 2015.
GPT-3 vs. 3M free-text trivia responses
A machine learning AI-esque robot challenges off on 156 trivia questions against tens of thousands of Water Cooler Trivia participants.
Why We Chose Redshift
How we decided on Redshift for giving our customers access to their raw data in SQL.
Apache Pinot 1.0
Introducing Apache Pinot 1.0 Release