Cloudflare Blog company blog
Perplexity is using stealth, undeclared crawlers to evade no-crawl directives
Perplexity is repeatedly modifying their user agent and changing IPs and ASNs to hide their crawling activity, in direct conflict with explicit no-crawl preferences expressed by websites.
- Points
- 1,315
- HN comments
- 748
- Submissions
- 1
- Cited by
- 11 commenters
AI & Machine LearningSociety, Policy & Culture2025
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| Aug 4, 2025 | Perplexity is using stealth, undeclared crawlers to evade no-crawl directives by rrampage | 1,315 | 748 |
Cited in comments
Linked from 12 Hacker News comments by 11 different people, from Aug 2025 to Jan 2026.
Most recent:
