The Two Dimensions of Worst-Case Training and Their Integrated Effect for Out-of-Domain Generalization
Abstract
Training with an emphasis on "hard-to-learn" components of the data has been proven as an effective method to improve the generalization of machine learning models, especially in the settings where robustness (e.g., generalization across distributions) is valued. Existing literature discussing this "hard-to-learn" concept are mainly expanded either along the dimension of the samples or the dimension of the features. In this paper, we aim to introduce a simple view merging these two dimensions, leading to a new, simple yet effective, heuristic to train machine learning models by emphasizing the worst-cases on both the sample and the feature dimensions. We name our method W2D following the concept of "Worst-case along Two Dimensions". We validate the idea and demonstrate its empirical strength over standard benchmarks.
Cite
Text
Huang et al. "The Two Dimensions of Worst-Case Training and Their Integrated Effect for Out-of-Domain Generalization." Conference on Computer Vision and Pattern Recognition, 2022. doi:10.1109/CVPR52688.2022.00941Markdown
[Huang et al. "The Two Dimensions of Worst-Case Training and Their Integrated Effect for Out-of-Domain Generalization." Conference on Computer Vision and Pattern Recognition, 2022.](https://mlanthology.org/cvpr/2022/huang2022cvpr-two/) doi:10.1109/CVPR52688.2022.00941BibTeX
@inproceedings{huang2022cvpr-two,
title = {{The Two Dimensions of Worst-Case Training and Their Integrated Effect for Out-of-Domain Generalization}},
author = {Huang, Zeyi and Wang, Haohan and Huang, Dong and Lee, Yong Jae and Xing, Eric P.},
booktitle = {Conference on Computer Vision and Pattern Recognition},
year = {2022},
pages = {9631-9641},
doi = {10.1109/CVPR52688.2022.00941},
url = {https://mlanthology.org/cvpr/2022/huang2022cvpr-two/}
}