Top posts

Every blog post that took off on Hacker News, ranked by the votes of its best submission. 4,225 posts.

Databases & Data
Any year
3279

Indexing a billion pages

It’s two years since we launched Mwmbl, the open source, non-profit search engine, on Boxing Day 2021. A good time to take stock of where we are and where we’re going. We’ve indexed over 100 million pages Thanks to our volunteers, who crawl the web using the Firefox extension and command line script, we’re crawling up…