Marginal Singularity, and the Benefits of Labels in Covariate-Shift
Abstract
We present new minimax results that concisely capture the relative benefits of source and target labeled data, under covariate-shift. Namely, we show that the benefits of target labels are controlled by a transfer-exponent $\gamma$ that encodes how singular Q is locally w.r.t. P, and interestingly allows situations where transfer did not seem possible under previous insights. In fact, our new minimax analysis - in terms of $\gamma$ - reveals a continuum of regimes ranging from situations where target labels have little benefit, to regimes where target labels dramatically improve classification. We then show that a recently proposed semi-supervised procedure can be extended to adapt to unknown $\gamma$, and therefore requests labels only when beneficial, while achieving minimax transfer rates.
Cite
Text
Kpotufe and Martinet. "Marginal Singularity, and the Benefits of Labels in Covariate-Shift." Annual Conference on Computational Learning Theory, 2018. doi:10.1214/21-aos2084Markdown
[Kpotufe and Martinet. "Marginal Singularity, and the Benefits of Labels in Covariate-Shift." Annual Conference on Computational Learning Theory, 2018.](https://mlanthology.org/colt/2018/kpotufe2018colt-marginal/) doi:10.1214/21-aos2084BibTeX
@inproceedings{kpotufe2018colt-marginal,
title = {{Marginal Singularity, and the Benefits of Labels in Covariate-Shift}},
author = {Kpotufe, Samory and Martinet, Guillaume},
booktitle = {Annual Conference on Computational Learning Theory},
year = {2018},
pages = {1882-1886},
doi = {10.1214/21-aos2084},
url = {https://mlanthology.org/colt/2018/kpotufe2018colt-marginal/}
}