Robustness to Spurious Correlation: A Comprehensive Review

Abstract

The persistence of spurious features in machine learning models remains a significant challenge. To address this issue, we identify several future directions that require attention. Firstly, we highlight the need for a new dataset that allows researchers to control the types and levels of spurious features, as this resource is currently lacking. Secondly, we emphasize the importance of addressing spurious features in natural language processing, where more attention is needed compared to vision-related tasks. We also stress the need for addressing spurious correlations at the core algorithmic level, rather than relying on complex, task-specific solutions that may not generalize well. Finally, we advocate for the development of weakly-supervised or unsupervised methods that reduce reliance on group labels, making the approaches more widely applicable. Our review aims to provide a comprehensive overview of existing work and guide future research in creating more robust machine learning models.