601
Faster PostgresSQL to BigQuery Transfers
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
New
The latest blog posts to take off on Hacker News. 1,839 posts.
Infrastructure & DevOps
Any year
602
603
Production Twitter on one machine? 100Gbps NICs and NVMe are fast
In this post I’ll attempt the fun stunt of designing a system that could serve the full production load of Twitter with most of the features intact on a single (very powerful) machine. I’ll start by showing off a Rust prototype of the core tweet distribution data structure handling 35x full load by fitting the hot set…
604
Ok, Cloudflare I am leaving
Music and blogging
605
General guidance when working as a cloud engineer
From the depths of the IT world, there are things that you only learn through experience.
606
2022 was the year of Linux on the Desktop?
It finally happened!
607
How much I’ve spent so far running my own Mastodon server on AWS
📖 You can read more of my posts on running your own Mastodon server right here: https://www.micahwalter.com/tag/mastodon/ I’ve been running my own single-user...
608
Nix and NixOS, my pain points
Posts and writings by Rémy Grünblatt 🍃
609
How Asus and a Microsoft Bug Almost Broke Remote Work
A couple of years after it happened I'm sharing this story about the intersection of an OS bug, a network hardware quirk, and a global pandemic. A chain…
610
Docker on MacOS is slow and how to fix it
Note 05/01/2025: Hello there, you can find an update version of this article here, with the latest findings, improvements and new tools. This is how a Docker container feels inside a macOS environment: lost at sea, far from the coast. With this article I will try to bring this poor container safely to the coast. TL;DR…
611
Ubuntu Snap update spoiled my World Cup Final
Goodbye Ubuntu, Goodbye Snaps I missed two goals in the World Cup Finals because of this. It’s not the first time Snap has been a problem, but it’s most definitely the last straw. What happened? I was happily watching the second half of the Fifa World Cup Final, using “ShowMax” – a South African paid … Continue…
612
Use a laptop as a 2nd display on Linux using FreeRDP
Production systems for ecommerce, finance, operations, and applied AI - built and operated for two decades.
613
614
The cloudy layers of modern-day programming
Is what we do development anymore?
615
FreeBSD on the Raspberry Pi
How to install FreeBSD on a Raspberry Pi, one step in building an Active Directory server with Samba and FreeBSD.
616
Snap updates happen without user consent
Recent versions of Ubuntu turned on a particularly annoying feature: intrusive system notifications that warn of the pending update of a Snap managed application. Here’s an example:
617
Self Hosting a Google Maps Alternative with OpenStreetMap
Self hosting a software stack to replace Google Maps on an unused gaming PC.
618
Why Twitter didn’t go down: From a real Twitter SRE
Twitter supposedly lost around 80% of its work force.
619
AWS and Blockchain
This week saw the cancellation of the Australian Stock Exchange’s long-running effort to build a blockchain-based trading system. Which, oddly, has me thinking of 2016, when AWS decided not to make a strategic investment in blockchain, with my input a contributing factor. It felt like a good story while it was…
620
The Modern Observability Problem
The observability problem - Why is the system exhibiting this behaviour?
621
Use a custom domain to send emails with Gmail using Cloudflare email routing
Edit: Updated with notes about revealing your main gmail address and deliverability, after this HN discussion
622
Fast CI with MicroVMs
I saw an opportunity to fix self-hosted runners for GitHub Actions. Actuated is now in pilot and aims to solve most if not all of the friction.
623
624
625
Scaling Mastodon is impossible
Now that Twitter is dying, what can replace it?
626
How to use your DSLR from 2008 as a webcam in 2022 (NixOS)
Why throw away your old Camera? In this guide we'll show you how simple it is to reuse your old Camera as a webcam using the NixOS linux distribution.
627
From Plex to Jellyfin Media Server
Why I uninstalled Plex Media Server after over a decade and migrated my movie and TV show collection to Jellyfin instead of Emby.
628
Tracing HTTP Requests with tcpflow
PagerDuty went off very early in the morning, and our hero cursed his bad luck and received the phone call. He made PagerDuty repeat the message twice, but couldn’t understand the message. He pressed the number to acknowledge the incident. He opened the MacBook and found the slack message from the PagerDuty bot more…
629
AWS IAM Roles, a tale of unnecessary complexity
This is going to be a highly opinionated blog post. I think AWS is great and use it daily, but their implementation of IAM is unnecessarily complicated.
630
Scaling Mastodon in the face of an exodus
TL;DR: Mastodon’s Sidekiq deferred execution jobs are the limiting factor for scaling federated traffic in a single-server or small cluster deployment. Sidekiq performance scales poorly under a single process model, and can be limited by database performance in a deployment of the default Dockerized configuration.