Building blocks of a scalable webcrawler.
In my last semester as a student, I had the chance of working for an awesome company ( Acquia ) on a very interesting project. It all started with a post over at Dries Buytaert's blog. He is the CTO and co-founder of Acquia (and Mollom), inventor of Drupal, open-source celebrity and also knows a thing or two about…