Geometry-Aware Collaborative Multi-Solutions Optimizer for Model Fine-Tuning with Parameter Efficiency
Abstract
We propose a framework grounded in gradient flow theory and informed by geometric structure that provides multiple diverse solutions for a given task, ensuring collaborative results that enhance performance and adaptability across different tasks. This framework enables flexibility, allowing for efficient task-specific fine-tuning while preserving the knowledge of the pre-trained foundation models. Extensive experiments across transfer learning, few-shot learning, and domain generalization show that our proposed approach consistently outperforms existing Bayesian methods, delivering strong performance with affordable computational overhead and offering a practical solution by updating only a small subset of parameters.
Cite
Text
Nguyen et al. "Geometry-Aware Collaborative Multi-Solutions Optimizer for Model Fine-Tuning with Parameter Efficiency." Advances in Neural Information Processing Systems, 2025.Markdown
[Nguyen et al. "Geometry-Aware Collaborative Multi-Solutions Optimizer for Model Fine-Tuning with Parameter Efficiency." Advances in Neural Information Processing Systems, 2025.](https://mlanthology.org/neurips/2025/nguyen2025neurips-geometryaware/)BibTeX
@inproceedings{nguyen2025neurips-geometryaware,
title = {{Geometry-Aware Collaborative Multi-Solutions Optimizer for Model Fine-Tuning with Parameter Efficiency}},
author = {Nguyen, Van-Anh and Le, Trung and Harandi, Mehrtash and Abbasnejad, Ehsan and Do, Thanh-Toan and Phung, Dinh},
booktitle = {Advances in Neural Information Processing Systems},
year = {2025},
url = {https://mlanthology.org/neurips/2025/nguyen2025neurips-geometryaware/}
}