New

The latest blog posts to take off on Hacker News. 2,538 posts.

Infrastructure & DevOps
Any year
433

Indexing a billion pages

It’s two years since we launched Mwmbl, the open source, non-profit search engine, on Boxing Day 2021. A good time to take stock of where we are and where we’re going. We’ve indexed over 100 million pages Thanks to our volunteers, who crawl the web using the Firefox extension and command line script, we’re crawling up…