Causal Information Splitting: Engineering Proxy Features for Robustness to Distribution Shifts
Abstract
Statistical prediction models are often trained on data that is drawn from different probability distributions than their eventual use cases. One approach to proactively prepare for these shifts harnesses the intuition that causal mechanisms should remain invariant between environments. Here we focus on a challenging setting in which the causal and anticausal variables of the target are unobserved. Leaning on information theory, we develop feature selection and engineering techniques for the observed downstream variables that act as proxies. We identify proxies that help to build stable models and moreover utilize auxiliary training tasks to extract stability-enhancing information from proxies. We demonstrate the effectiveness of our techniques on synthetic and real data.
Cite
Text
Mazaheri et al. "Causal Information Splitting: Engineering Proxy Features for Robustness to Distribution Shifts." Uncertainty in Artificial Intelligence, 2023.Markdown
[Mazaheri et al. "Causal Information Splitting: Engineering Proxy Features for Robustness to Distribution Shifts." Uncertainty in Artificial Intelligence, 2023.](https://mlanthology.org/uai/2023/mazaheri2023uai-causal/)BibTeX
@inproceedings{mazaheri2023uai-causal,
title = {{Causal Information Splitting: Engineering Proxy Features for Robustness to Distribution Shifts}},
author = {Mazaheri, Bijan and Mastakouri, Atalanti and Janzing, Dominik and Hardt, Michaela},
booktitle = {Uncertainty in Artificial Intelligence},
year = {2023},
pages = {1401-1411},
volume = {216},
url = {https://mlanthology.org/uai/2023/mazaheri2023uai-causal/}
}