New

The latest blog posts to take off on Hacker News. 4,389 posts.

Systems & Performance
Any year
1895

The Container Throttling Problem

This is an excerpt from an internal document David Mackey and I co-authored in April 2019. The document is excerpted since much of the original doc was about comparing possible approaches to increasing efficency at Twitter, which is mostly information that's meaningless outside of Twitter without a large amount of…

1913

Fast CSV Processing with SIMD

I recently learned of csvquote , a tool that encodes troublesome CSV characters such that unix tools can correctly process them. It reverses the encoding at the end of the pipeline, recovering the original input. The original implementation handles CSV quotes using the straightforward, naive method. However, there’s a…

1914

Firefox's Optimized Zip Format

This post is about minimizing amount of disk IO and CPU overhead when reading Zip files. I recently saw an article about a new format that was faster than zip. This is quite surprising as to my mind, zip is one of the most flexible and low-overhead formats I’ve encountered. Some googling showed me that over past 11…

1919

Decompressing a Gzip File by Hand

2024 Update: Part 2 is out! Let’s make a gzipped file and see what’s in it. We’ll keep it simple: just write 8 ‘a’s to a file. $ echo "aaaaaaaa" > test.out $ xxd test.out 00000000: 6161 6161 6161 6161 0a aaaaaaaa. As we can see, our file is 9 bytes long. We have 8 ‘a’ bytes written, plus a Line Feed (LF) character…