Not on this site, but still likely to be of interest, is my Tarsnap online backup service .
Visit daemonology.net → On Hacker News Nov 2008 – Oct 2026
Security & Privacy 22Tech Industry & Business 21Infrastructure & DevOps 16Systems & Performance 12Software Engineering 7Life, Learning & Writing 6
When the Tarsnap EC2 instance came back online about an hour later, I found that the abrupt loss of power had resulted in filesystem corruption. As I explained in a blog post in December 2008, Tarsnap stores all user data in Amazon S3, but keeps metadata cached in EC2 . While I could not see any evidence that the…
There's just one small problem: The way POSIX has defined close(2) is completely and utterly broken.
Ever since the first EC2 instances launched — the ones which were retrospectively named "m1.small" — Amazon has relied on the Xen hypervisor. No longer: C5 instances use KVM. This sounds like it would be a problem, but in fact that change didn't bother FreeBSD at all: Now that everything uses hardware-based paging…
As I've blogged about before, the Tarsnap server code synthesizes a log-structured filesystem on top of Amazon S3 . As a result of this design, for each block of data stored on Tarsnap, the server code needs to keep track of two key-value pairs: First, to map a 64-bit machine number and a 256-bit block ID to a 64-bit…
This tension of what customers are asking for vs what customers really need shows up in areas like resilience. Amazon's "Well-Architected Framework" strongly exhorts customers to avoid building production workloads in a single Availability Zone — but Amazon's cross-AZ bandwidth pricing is painful, and Amazon doesn't…
What does it mean for a baby monitor to be safe? Some things are clearly essential: It should not explode; it should not release significant quantities of dangerous gasses; if it's designed to be placed within reach of a baby, it should not pose an electrocution or choking hazard. Less obviously, if it has data or…
I've been excited about Stripe ever since one of the founders, Patrick Collison, emailed me in 2009 to say that he was "trying to build a credit-card processor that doesn't suck". My immediate reaction: "I think that might be the most ambitious plan I've ever heard from someone associated with YC ". When Patrick went…
A few months ago, after my System76 Galago Pro had its second experience with a dead/swelling batery, I decided that it was time to replace it. On February 15th, I ordered a Dell Latitude 7390.
There was a joke at the time that although the National Security Agency had opened up in recent years and started taking their defensive role more seriously, the one aspect of cryptography which they would never discuss publicly was side channel attacks — and this itself constituted a side channel, revealing the…
One consequence of these unsolicited security scans is that — since they feed a variety of inputs to forms, including the account creation form — I see a lot of obviously fake signup attempts (alas, none yet from the world's most obviously fake domain name ). These are harmless, since the signup code sends out a…
I tried building FreeBSD AMIs for the 32-bit EC2 instance types the same way as I did the 64-bit instances — by defenestrating Windows AMIs — but there was a catch: They wouldn't boot. In fact, when I tried this six months ago, they not only didn't boot, but they didn't even produce any console output to help me…
One of the unusual aspects of my online backup startup is that I'm a sole founder. Despite such commentary as Gabriel Weinberg's Will single founders please stand up? and Ray Grieselhuber's The Royal We: Single Founder Startups , single-founder startups are still unusual and carry with them some stigma — Paul Graham…
While I've made engineers inside Amazon aware of all of these at various times, I think it's time to post my wishlist here — both so that a wider audience inside Amazon can hear more about these, and so that the FreeBSD community (especially the people who are financially supporting my work ) can see what I'm aiming…
Enter magic getopt . Via a little bit of macro magic, the above options-handling code turns into this:
Encrypting data: Use AES in CTR (Counter) mode, and append an HMAC . AES is about as standard as you can get, and has done a good job of resisting cryptologic attacks over the past decade. Using CTR mode avoids the weakness of ECB mode, the complex (and bug-prone) process of padding and unpadding of partial blocks (or…
Four decades ago, when the Berkeley Internet Name Domain Server launched, it stored DNS records in "zone files". You could edit these files in vi , but BIND had to be told to reload them before your changes took effect. Later, other DNS servers came along, like Daniel Bernstein's tinydns, which served records directly…
You're going to want to make some changes to what goes into that disk image later, but for now just build a disk image.
Most bugs are straightforward, but occasionally one comes up which is not so clear in its impact. Such is the case with a bug which is fixed in tarsnap 1.0.36 . This bug causes the NUL string termination byte to overflow the heap-allocated buffer used for paths of objects examined as tarsnap traverses a directory…
Once I finished writing ec2-boot-bench, the natural next step was to run some tests — in particular, to see how FreeBSD compared to other operating systems used in EC2. I used the c5.xlarge instance type and tested FreeBSD releases since 11.1-RELEASE (the first FreeBSD release which can run on the c5.xlarge instance…
As far as FreeBSD is concerned, Graviton 3 is mostly just a faster version of the Graviton 2: Most things "just work", and the things which don't work on Graviton 2 — hotplug devices and cleanly shutting down an instance via the EC2 API — also don't work on Graviton 3. (Want to help get these fixed? Sponsor my work on…
[EDIT 2010-03-11 18:00 I mention this below, but to place my conflict up front: I'm the author of the Tarsnap secure online backup service, which is in some ways a competitor to ZumoDrive.]
The key detail to understand about the work in the Joux papers is that it is limited to solving discrete logarithm problems over fields of small characteristic . As interesting as the work is, mathematically, it should not be a great surprise to cryptographers: We've known for over a decade that small-characteristic…
One of the most common questions I receive about tarsnap is this: What guarantee can I provide that the tarsnap service will continue to operate? This is a reasonable concern; after all, with services like Omnidrive and HP Upline shutting down, what guarantee is there that tarsnap won't also shut down? Until recently…
I am speaking today at BSDCan'09 about my work on the scrypt key derivation function. In the context of hardware brute-force attacks, scrypt is thousands of times more secure than existing "best practice" solutions such as bcrypt and PBKDF2; in fact, under reasonable assumptions it is provably as strong as possible…
First things first: Why use a composition of encryption and MAC instead of a single primitive which achieves both? Because people are very good at writing bad code. In my experience on the FreeBSD security team, writing dozens of security advisories, there are three types of code which are especially prone to have…
Much of the work done by the FreeBSD Foundation is done behind the scenes, but the importance of the work they do is undeniable. They sponsor a range of important FreeBSD development which would likely not get done otherwise; in the past year this has included
Complexity can be thought of as a type of code smell : It doesn't necessarily imply that there is a problem, but the presence of complexity is very strongly correlated with the presence of security vulnerabilities. In the design and construction of secure systems, it is important to not only consider mistakes which…
Tarsnap's paranoia also extends to the client-server protocol. With most online backup services, you're lucky if they use SSL; I avoided using SSL because I wasn't satisfied with its security. Obtaining a fraudulent SSL certificate is too easy (read: possible) for an attacker, and relying on external certification…
For lists of FreeBSD AMI IDs, please consult the appropriate FreeBSD release announcement; you can also launch FreeBSD through the AWS Marketplace .