1171
1.1B Taxi Trips on BrytlytDB 2.0
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
New
The latest blog posts to take off on Hacker News. 1,888 posts.
1172
1173
A/B test of banner ads vs. traffic
9 months of daily A/B-testing of Google AdSense banner ads on Gwern.net indicates banner ads decrease total traffic substantially, possibly due to spillover effects in reader engagement and resharing.
1174
How should you build a high-performance column store for the 2020s?
Though most relational databases (like MySQL) are "row oriented", in that they keep rows stored together... experience has taught us that pivoting the data arrow so that columns, and not rows, are stored together can be beneficial. This is an old observation that most experienced programmers know about as the…
1175
Redis Streams and the Unified Log
Building a log-based architecture that’s fast, efficient, and resilient on the new stream data structure in Redis.
1176
1177
Connecting to Google Sheets with Python
I wrote up my experience using Python to process data from a shared Google Sheets spreadsheet
1178
A Minimalist Guide to SQLite
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
1179
Do we need a third Apache project for columnar data representation?
Apache Parquet and Apache ORC have become a popular file formats for storing data in the Hadoop ecosystem. Their primary value proposition ...
1180
1181
The often-overlooked random forest kernel
Ryan Marcus, assistant professor at the University of Pennsylvania. Using machine learning to build the next generation of data systems.
1182
Implementing Stripe-Like Idempotency Keys in Postgres
Building resilient services by identifying foreign state mutations and grouping local changes into restartable atomic phases so that every request can be driven to completion.
1183
The Upspin manifesto: On the ownership and sharing of data (2014)
Here follows the original "manifesto" from late 2014 proposing the idea for what became Upspin . The text has been lightly edited to remove...
1184
1185
Getting the Most Out of Sqlite3 with Python
Simplex Sigillum Veri
1186
1187
Efficient pagination of a SQL table with 100M records
Compilation of my favorite SQL queries after a decade of dealing with relational databases. Today we talk about scanning a table with 100 million records.
1188
Unbalanced reads from SSDs in software RAID mirrors in Linux
You're probably reading this page because you've attempted to access some part of my blog (Wandering Thoughts) or CSpace , the wiki thing it's part of. Unfortunately you're using a browser version that my anti-crawler precautions consider suspicious, most often because it's too old (most often this applies to versions…
1189
Analysing C# code on GitHub with BigQuery
Just over a year ago Google made all the open source code on GitHub available for querying within BigQuery and as if that wasn’t enough you can run a terabyte of queries each month for free!
1190
Dear Postgres
Dear Postgres, I’ve always felt an affinity for you in my 9 years of working with you. I know others have known you longer, but that doesn’t mean they love you more. Years ago when others complained about your rigidness or that you weren’t as accommodating as others I found solace in your steadfast values: Don’t lose…
1191
A Data-Driven Guide to Becoming a Consistent Billionaire
Did You Really Think All Billionaires Were the Same? Recently, I became a bit obsessed with the one percent of the one percent – Billionaires. I was intrigued when I stumbled on articles telling us…
1192
How I Reduced My DB Server Load by 80%
Database load can be a silent performance killer. I’ve been optimizing the query performance of a web app I run designed to get people involved in open sourc...
1193
Implementing faceted search with Django and PostgreSQL
I’ve added a faceted search engine to this blog, powered by PostgreSQL. It supports regular text search (proper search, not just SQL“like” queries), filter by tag, filter by date, filter …
1194
A Decade of Dynamo
Today marks the 10-year anniversary of Amazon's Dynamo. Here are some reflections on ten years of NoSQL databases
1195
1196
1197
1198
Wikipedia graph mining: dynamic structure of collective memory
Wikipedia graph dynamic pattern discovery. Dynamic graph analysis in GraphX. Collective memory research. Gephi graph visualization.
1199
Visualizing Your Ruby Heap
In a previous post, I wrote a bit about how Ruby objects are laid out in memory. Today we’ll use that information to write a program that will allow us to take a Ruby heap dump and visualize the layout and fragmentation of that heap. Ruby Object Layout Recap Just as a recap, Ruby objects are fixed width. That is…
1200
Stream VByte: breaking new speed records for integer compression
In many information systems, we work with arrays of integers. For example, maybe you need to keep track of which records in a database contain a given value. This sort of mapping can be expressed as an array of integers. These arrays can end up taking up a large fraction of your memory or disk … Continue reading…