211
Continuous batching to increase LLM inference throughput and reduce p50 latency
In this blog, we discuss continuous batching, a critical systems-level optimization that improves both throughput and latency under load for LLMs.
New
The latest blog posts to take off on Hacker News. 555 posts.
212
213
Emulating x86 on X64 on Aarch64
Emulating win32 on Mac using Apple's x64 emulator.
214
Ode to the M1
It is not often I am blown away by a piece of hardware. My Hall of Fame features the 1991 Super Nintendo, the 1995 3dfx1, and the 2010 Intel's Serie 520 SSD [1] . This past month my 2021 MacBook Pro M1 [2] joined the club.
215
216
Fedora Asahi Remix first impressions
My personal website and blog
217
What is gained and lost with 63-bit integers? (2014)
Almost every programming language uses 64-bit integers on typical modern Intel machines. OCaml uses a special 63-bit representation. How does it affect OCaml?
218
Wendelstein 7-X: Gigajoule energy turnover generated for eight minutes
After successful recommissioning in autumn 2022, the Greifswald nuclear fusion experiment has surpassed an important target.
219
Squeeze the hell out of the system you have
When complexity leaps are on the table, there’s usually also an opportunity to squeeze some extra juice out of the system you have. By tweaking the workload, tuning performance, or supplement…
220
Graceful behavior at capacity
Suppose we’ve got a service. We’ll gloss over the details for now, but let’s stipulate that it accepts requests from the outside world, and takes some action in response. Maybe those requests are HTTP requests, or RPCs, or just incoming packets to be routed at the network layer. We can get more specific later. What…
221
Linux guide for power users (2022)
A guide to help you setup your Linux work machine and configure it for a power users workflow
222
Gnu/Hurd strikes back: How to use the legendary OS in a (somewhat) practical way
How to use the legendary OS in a (somewhat) practical way
223
Simple Networking in Linux
Industrial production of readable insanity
224
Snappy UIs with WebAssembly and Web Workers
Our web app allows users can change the length of a song or find loops present in it for infinite listening, remixing, or for their next next video edit or performance. After uploading a song, there is an initial server-side analysis step after which...
225
Why libvirt supports only 14 PCIe hotplugged devices on x86-64
You might have used libvirt , a Linux daemon that provides an abstraction and management layer for virtualization software. This daemon allows you to submit an XML file containing a description of a VM, disk image or network. libvirt is then able to configure, start it and keep it running.
226
System settings that aren’t in System Settings
Where would you set the default call handler for macOS? And where in System Settings do you set the default web browser?
227
Systemd auto-restarts of units can hide problems from you
You're probably reading this page because you've attempted to access some part of my blog (Wandering Thoughts) or CSpace , the wiki thing it's part of. Unfortunately you're using a browser version that my anti-crawler precautions consider suspicious, most often because it's too old (most often this applies to versions…
228
Can you trust a compiler to optimize your code?
More or less the title this time, but first, a story about SIMD. There are three levels of understanding how SIMD works (well, at least I am level 3 at the moment):
229
x86 is dead, long live x86
The last couple of years have been quite revolutionary in the Silicon industry as a whole. With the resurgence of horizo
230
The shell and its crappy handling of whitespace
From the highly eclectic blog of Mark Dominus
231
A dive into the AMD driver workflow
I ended up getting a response from high level people at AMD. It was still very light on any real technical information, but it did include some great phrases like “I am able to replicate the issues you are facing” and some mockable phrases like “We are hoping that this will improve your perception of AMD products and…
232
On the future of free long term support for Linux distributions
You're probably reading this page because you've attempted to access some part of my blog (Wandering Thoughts) or CSpace , the wiki thing it's part of. Unfortunately you're using a browser version that my anti-crawler precautions consider suspicious, most often because it's too old (most often this applies to versions…
233
Faster filesystem access with Directfs
Directfs is now the default in runsc. This feature gives gVisor’s application kernel (the Sentry) secure direct access to the container filesystem, avoiding expensive round trips to the filesystem gofer. Learn more about this feature in the following blog that was originally posted on Google Open Source Blog .
234
Building and operating a pretty big storage system called S3
Three distinct perspectives on scale that come along with building and operating a storage system the size of S3.
235
SQLite Begin Concurrent
Usually, SQLite allows at most one writer to proceed concurrently. The BEGIN CONCURRENT enhancement allows multiple writers to process write transactions simultanously if the database is in "wal" or "wal2" mode, although the system still serializes COMMIT commands.
236
Rethinking Window Management
Window management is one of those areas I'm fascinated with because even after 50 years, nobody's fully cracked it yet. Ever since the dawn of time we've relied on the window metaphor as the primary way of multitasking on the desktop. In this metaphor, each app can spawn one or more rectangular windows, which are...
237
Memory Copy Hunting
I have a super power. I want to share it in this blog post so that you, my dear reader, also have it!
238
Practical comparison of ARM compilers
My inspiration for this article was this thread in r/embedded . There are still many myths and misconception when it comes to the simple questions "Which compiler should I use?" or "Which compiler is the best?". I will try to share my practical experience with GCC, Clang and IAR on Cortex-M ( and Cortex-M only ). This…
239
Advanced Performance Extensions (APX)
These extensions expand the x86 instruction set with access to registers and features that improve performance.
240
Zenbleed
If you remove the first word from the string "hello world" , what should the result be? This is the story of how we discovered that the answer could be your root password!