Learning Pareto Manifolds in High Dimensions: How Can Regularization Help?
Abstract
Simultaneously addressing multiple objectives is becoming increasingly important in modern machine learning. At the same time, data is often high-dimensional and costly to label. For a single objective such as prediction risk, conventional regularization techniques are known to improve generalization when the data exhibits low-dimensional structure like sparsity. However, it is largely unexplored how to leverage this structure in the context of multi-objective learning (MOL) with multiple competing objectives. In this work, we discuss how the application of vanilla regularization approaches can fail, and propose a two-stage MOL framework that can successfully leverage low-dimensional structure. We demonstrate its effectiveness experimentally for multi-distribution learning and fairness-risk trade-offs.
Cite
Text
Wegel et al. "Learning Pareto Manifolds in High Dimensions: How Can Regularization Help?." Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, 2025.Markdown
[Wegel et al. "Learning Pareto Manifolds in High Dimensions: How Can Regularization Help?." Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, 2025.](https://mlanthology.org/aistats/2025/wegel2025aistats-learning/)BibTeX
@inproceedings{wegel2025aistats-learning,
title = {{Learning Pareto Manifolds in High Dimensions: How Can Regularization Help?}},
author = {Wegel, Tobias and Kovačević, Filip and Tifrea, Alexandru and Yang, Fanny},
booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics},
year = {2025},
pages = {4591-4599},
volume = {258},
url = {https://mlanthology.org/aistats/2025/wegel2025aistats-learning/}
}