Adaptive Instrument Design for Indirect Experiments

Abstract

Indirect experiments provide a valuable framework for estimating treatment effects in situations where conducting randomized control trials (RCTs) is impractical or unethical. Unlike RCTs, indirect experiments estimate treatment effects by leveraging (conditional) instrumental variables, enabling estimation through encouragement and recommendation rather than strict treatment assignment. However, the sample efficiency of such estimators depends not only on the inherent variability in outcomes but also on the varying compliance levels of users with the instrumental variables and the choice of estimator being used, especially when dealing with numerous instrumental variables. While adaptive experiment design has a rich literature for \textit{direct} experiments, in this paper we take the initial steps towards enhancing sample efficiency for \textit{indirect} experiments by adaptively designing a data collection policy over instrumental variables. Our main contribution is a practical computational procedure that utilizes influence functions to search for an optimal data collection policy, minimizing the mean-squared error of the desired (non-linear) estimator. Through experiments conducted in various domains inspired by real-world applications, we showcase how our method can significantly improve the sample efficiency of indirect experiments.

Cite

Text

Chandak et al. "Adaptive Instrument Design for Indirect Experiments." International Conference on Learning Representations, 2024.

Markdown

[Chandak et al. "Adaptive Instrument Design for Indirect Experiments." International Conference on Learning Representations, 2024.](https://mlanthology.org/iclr/2024/chandak2024iclr-adaptive/)

BibTeX

@inproceedings{chandak2024iclr-adaptive,
  title     = {{Adaptive Instrument Design for Indirect Experiments}},
  author    = {Chandak, Yash and Shankar, Shiv and Syrgkanis, Vasilis and Brunskill, Emma},
  booktitle = {International Conference on Learning Representations},
  year      = {2024},
  url       = {https://mlanthology.org/iclr/2024/chandak2024iclr-adaptive/}
}