Leveraging Covariates in Randomized Experiments Guided by Causal Structure
by
Ali Tafti
Associate Professor of Information and Decision Sciences
College of Business, University of Illinois at Chicago
Friday, November 9, 2018
10:30 AM – noon
Speakman Hall Suite 200
Abstract
Researchers using randomized controlled trial (RCT) experiments often subgroup or condition on auxiliary variables that are not the randomized treatment variables. There are many good reasons to condition on auxiliary variables—also referred to as control variables or covariates— in randomized experiments. In particular, designing and conducting RCTs is costly to researchers and subjects. Therefore, it’s important to derive greater value from RCTs that are conducted; measuring not just the average treatment effect (ATE), but also finding more nuanced insights about the underlying theoretical mechanisms and generalizing the inferences. Unfortunately, there are many confusing and even contradictory guidelines on the use of subgroups or auxiliary variables in RCTs. For example, the common wisdom is that post-treatment variables (i.e. those that are ex-post to the treatment variable) should not be conditioned on. However, such variables can provide valuable information, and can in many cases be properly utilized. Using causal diagrams, and applying a few simple rules based upon Judea Pearl’s causal diagramming framework, we explain how researchers can leverage covariates without biasing their causal inferences. We provide guidelines for using subgroups and auxiliary variables in randomized experiments, focusing on some well-known digital experiments featured in the Information Systems literature.