Anton Vodonosov personal blog
A robots.txt Problem
To prevent a part of our web application from being scanned by search engines and other web crawlers, we add a robots.txt like User-agen...
- Points
- 158
- HN comments
- 50
- Submissions
- 1
- Cited by
- 0 commenters
Web & FrontendInfrastructure & DevOps2022
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| Aug 15, 2022 | A robots.txt Problem by avodonosov | 158 | 50 |
