Colin Percival

The EC2 firewall is broken

As most standards-aware network administrators know, blocking ICMP is evil. Sure, there are certain ICMP packets which should be blocked; but to quote RFC 2979 :

EC2's most dangerous feature

A review for readers unfamiliar with this feature: Amazon IAM ( Identity and Access Management ) is a service which allows for the creation of access credentials which are limited in scope; for example, you can have keys which can read objects from Amazon S3 but cannot write any objects. IAM Roles for EC2 are a…

FreeBSD/EC2 lives

There are some caveats to this. First, at the moment only FreeBSD 9.0-CURRENT can run under EC2; I haven't merged bug fixes back to the stable branches. Second, at the moment FreeBSD only runs on t1.micro instances, for reasons I can't discuss (NDA) but hope will be resolved soon. Third, this code hasn't received very…

The FreeBSD/Firecracker Platform

I started working on this on June 20th mainly out of curiosity: I had heard that Firecracker had PVH boot support (which was in fact mistaken!) and I knew that FreeBSD could boot in PVH mode from Xen, so I wondered just how hard it would be to get FreeBSD up and running. Not impossible, as it turned out, but a bit…

Tarsnap: No heartbleed here

There's a few reasons for this. First, the Tarsnap client-server protocol does not use TLS; instead, it uses a protocol which is about 100x simpler (in terms of the number of pages of documentation required to define it). This means there's less code and far fewer edge cases where bugs can lurk without being triggered…

Why Tarsnap doesn't use Glacier

On the surface, Tarsnap sounds like the perfect use case for Glacier. Out of every TB of data stored on Tarsnap, in any given month approximately 100 GB of data will be deleted, while only 33 GB of data will be downloaded; in other words, Tarsnap is very much a "write once, read maybe " storage system. Tarsnap's…

Tarsnap outage post-mortem

When the Tarsnap EC2 instance came back online about an hour later, I found that the abrupt loss of power had resulted in filesystem corruption. As I explained in a blog post in December 2008, Tarsnap stores all user data in Amazon S3, but keeps metadata cached in EC2 . While I could not see any evidence that the…

FreeBSD/EC2 on C5 instances

Ever since the first EC2 instances launched — the ones which were retrospectively named "m1.small" — Amazon has relied on the Xen hypervisor. No longer: C5 instances use KVM. This sounds like it would be a problem, but in fact that change didn't bother FreeBSD at all: Now that everything uses hardware-based paging…

Why Tarsnap won't use DynamoDB

As I've blogged about before, the Tarsnap server code synthesizes a log-structured filesystem on top of Amazon S3 . As a result of this design, for each block of data stored on Tarsnap, the server code needs to keep track of two key-value pairs: First, to map a 64-bit machine number and a 256-bit block ID to a 64-bit…

FreeBSD now available on all EC2 instance types

I tried building FreeBSD AMIs for the 32-bit EC2 instance types the same way as I did the 64-bit instances — by defenestrating Windows AMIs — but there was a catch: They wouldn't boot. In fact, when I tried this six months ago, they not only didn't boot, but they didn't even produce any console output to help me…

My EC2 Wishlist

While I've made engineers inside Amazon aware of all of these at various times, I think it's time to post my wishlist here — both so that a wider audience inside Amazon can hear more about these, and so that the FreeBSD community (especially the people who are financially supporting my work ) can see what I'm aiming…

Launching Route 53 Files

Four decades ago, when the Berkeley Internet Name Domain Server launched, it stored DNS records in "zone files". You could edit these files in vi , but BIND had to be told to reload them before your changes took effect. Later, other DNS servers came along, like Daniel Bernstein's tinydns, which served records directly…

How to port your OS to EC2 (2018)

You're going to want to make some changes to what goes into that disk image later, but for now just build a disk image.

FreeBSD/EC2 boot performance over time

EC2 Boot Time Benchmarking

Once I finished writing ec2-boot-bench, the natural next step was to run some tests — in particular, to see how FreeBSD compared to other operating systems used in EC2. I used the c5.xlarge instance type and tested FreeBSD releases since 11.1-RELEASE (the first FreeBSD release which can run on the c5.xlarge instance…

FreeBSD on EC2 status

For lists of FreeBSD AMI IDs, please consult the appropriate FreeBSD release announcement; you can also launch FreeBSD through the AWS Marketplace .