3031
Scaling Secret: Real-Time Chat
(This is the first of a series of technical posts I intend to write, and my goal is to help other engineers learn from the experience of…
Topics /
Systems & Performance
The most popular blog posts about systems & performance on Hacker News. 4,389 posts.
3032
3033
Unwinding the Stack the Hard Way
Tutorial on how to make a stack trace or backtrace manually unwinding the stack using ELF’s .eh_frame and call frame information (CFI)
3034
Linux commands for advanced hardware and system info
Sometimes you need info about hardware, and you probably lost your invoice, spec list or a password to a store website. Maybe you did an upgrade and this info isn't accurate anymore. It's an easy case for home users, but what to do is you have many machines
3035
Debugging random slow writes in PostgreSQL
Understanding & mitigating randomly slow writes in PostgreSQL
3036
Tracking Java native memory with JDK flight recorder
As regular readers of this blog will now, JDK Flight Recorder (JFR) is one of my favorite tools of the Java platform. This low-overhead event recording engine built into the JVM is invaluable for …
3037
Non-blocking transactional atomicity
tl;dr: You can perform non-blocking multi-object atomic reads and writes across arbitrary data partitions via some simple multi-versioning and by storing metadata regarding related items.
3038
3039
3040
Apache Spark: The Next Big Data Thing?
Apache Spark is generating quite some buzz right now. Databricks, the company founded to support Spark raised $14M from Andreessen Horowitz, Cloudera has decided to fully support Spark, and others chime in that it's the next big thing. So I thought it's high time I took a look to get an understanding what the whole…
3041
Large Transformer Model Inference Optimization
[Updated on 2023-01-24: add a small section on Distillation.] Large transformer models are mainstream nowadays, creating SoTA results for a variety of tasks. They are powerful but very expensive to train and use. The extremely high inference cost, in both time and memory, is a big bottleneck for adopting a powerful…
3042
Simulating RAM in Clojure
“Computers are all made out of logic gates”. We’ve heard that saying before. We also have a sense that logic gates are very simple machines, analogous to light switches even. This raises the question: how exactly do kind-of-light-switches come together to form computers ? How does “storing a variable” or “calling a…
3043
1.1B taxi rides benchmark on the GPU- and PostgreSQL-powered BrytlytDB
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
3044
Assembly Language: Still Relevant Today
First, no, I don't do every thing in assembly. I use other languages too. Assembly language does have its place though. Second, a few of my points may not apply to highly complex processors; for example, self-modifying code may not work on a processor with an instruction cache .
3045
OS X 10.10.3 seems to have done something bad to wifi drivers
As a PSA (public service announcement), I'm reporting here that updating your Yosemite system to 10.10.3 is incredibly toxic if you use WiFi...
3046
Let's consign CAP to the cabinet of curiosities
Brewer’s CAP theorem, and Gilbert and Lynch’s formalization of it , is the first introduction to hard trade-offs for many distributed systems engineers. Going by the vast amounts of ink and bile spent on the topic, it is not unreasonable for new folks to conclude that it’s an important, foundational, idea.
3047
3048
How to build a Linux-based wireless router out of spare parts (1998)
This is a brief mini-howto on how to build a linux based wireless router out of what amounts to spare parts, using the linux router project's single disk router software. The total cost of each end of the router, including antennas and cabling, is less than 1300 US dollars, which is a considerable cost savings over…
3049
OpenSSH taking minutes to become available, booting takes half an hour (2018)
So, your machine now needs minutes to boot before you can ssh in where it used to be seconds before the Debian Buster update?
3050
A Sketch of the Biggest Idea in Software Architecture (2022)
Narrow waists: an idea in software architecture that relates to networking, operating systems, language design, compilers, and distributed systems.
3051
Is parallel programming hard, and, if so, what can you do about it?
There is a new release of Is Parallel Programming Hard, And, If So, What Can You Do About It? . This release features a number of formatting and build-system improvements by the indefatigible Akira Yokosawa. On the formatting side, we have listings automatically generated from source code, clever…
3052
3053
Fiber in C++: Understanding the Basics
Fiber, a less known concept compared with coroutine, is a pretty powerful addition to cooperative multitasking. As a graphics programmer in game industry, I totally appreciate the great flexibility that fiber brings on the table. As a matter of fact, I feel the tech is a little bit underappreciated due to the lack of…
3054
3055
Linux's fsync() woes are getting some attention
In two weeks, I'm headed to LSF/MM and the Linux Collaboration Summit , by invitation of some Linux kernel hackers, to discuss how the Linu...
3056
The best worst hack that saved our bacon
No-one really likes engineering war stories, but this one's relevant because there's a moral to it. I've talked before about defining technical debt as technical decisions that provide immediate value, but with long-term negative impact if they aren't cleaned up. Sometimes introducing technical debt
3057
Load testing is hard and the tools are not great
If you're building an application that needs to scale—and we all tell ourselves that we are—then at some point you have to figure out if it does or not. This is where load testing comes in: if you want to see whether or not your application can handle scale, just generate scale and see if it can handle it! It sounds…
3058
3059
Service mesh use cases (2020)
Service mesh is a blazing hot topic in software engineering right now and rightfully so. I think it’s extremely promising technology and I’d love to see it widely adopted (in cases where it truly makes sense). Yet it remains shrouded in mystery for many people and even those who are familiar with it have trouble…
3060
Building a high performance JSON parser
JSON is important, damn near everything that we do as programmers or operators involves JSON at some point. JSON decoding is expensive, if your product talks JSON then performance of marshalling data in and out of JSON is important. This is a talk about designing an efficient replacement for encoding/json.Decoder .