Promoting Ensemble Diversity with Interactive Bayesian Distributional Robustness for Fine-Tuning Foundation Models
Abstract
We introduce Interactive Bayesian Distributional Robustness (IBDR), a novel Bayesian inference framework that allows modeling the interactions between particles, thereby enhancing ensemble quality through increased particle diversity. IBDR is grounded in a generalized theoretical framework that connects the distributional population loss with the approximate posterior, motivating a practical dual optimization procedure that enforces distributional robustness while fostering particle diversity. We evaluate IBDR’s performance against various baseline methods using the VTAB-1K benchmark and the common reasoning language task. The results consistently show that IBDR outperforms these baselines, underscoring its effectiveness in real-world applications.
Cite
Text
Pham et al. "Promoting Ensemble Diversity with Interactive Bayesian Distributional Robustness for Fine-Tuning Foundation Models." Proceedings of the 42nd International Conference on Machine Learning, 2025.Markdown
[Pham et al. "Promoting Ensemble Diversity with Interactive Bayesian Distributional Robustness for Fine-Tuning Foundation Models." Proceedings of the 42nd International Conference on Machine Learning, 2025.](https://mlanthology.org/icml/2025/pham2025icml-promoting/)BibTeX
@inproceedings{pham2025icml-promoting,
title = {{Promoting Ensemble Diversity with Interactive Bayesian Distributional Robustness for Fine-Tuning Foundation Models}},
author = {Pham, Ngoc-Quan and Truong, Tuan and Tran, Quyen and Nguyen, Tan Minh and Phung, Dinh and Le, Trung},
booktitle = {Proceedings of the 42nd International Conference on Machine Learning},
year = {2025},
pages = {49259-49276},
volume = {267},
url = {https://mlanthology.org/icml/2025/pham2025icml-promoting/}
}