1
AI crawlers need to be more respectful
We talk a bit about the AI crawler abuse we are seeing at Read the Docs, and warn that this behavior is not sustainable.
Read the Docs is a documentation building and hosting platform aimed at helping developers creating documentation from code with versioned documentation, integrated search, pull request previews and more.
Visit about.readthedocs.com → RSS feedOn Hacker News Jul 2024 – Sep 2026
Society, Policy & Culture 1Security & Privacy 1Infrastructure & DevOps 1AI & Machine Learning 1
We talk a bit about the AI crawler abuse we are seeing at Read the Docs, and warn that this behavior is not sustainable.
A deep dive into the massive June 2026 DDoS attack on Read the Docs, the attack patterns we observed, and the mitigations and defenses we used to defend our infrastructure.