Top posts

Every blog post that took off on Hacker News, ranked by the votes of its best submission. 31,720 posts.

Any topic
Any year
25496

Indexing a billion pages

It’s two years since we launched Mwmbl, the open source, non-profit search engine, on Boxing Day 2021. A good time to take stock of where we are and where we’re going. We’ve indexed over 100 million pages Thanks to our volunteers, who crawl the web using the Firefox extension and command line script, we’re crawling up…