Check what your site tells AI crawlers: botpolicy.org measures your robots.txt against 166 known AI crawlers.
- Botpolicy.org checks your robots.txt against 166 known AI crawlers, shows what is still getting through, and covers TDM reservations and llms.txt as additional layers.
- The tool is honest about limits: robots.txt is a request, not a lock, and only TDM reservations carry legal weight under EU copyright law.
