Improving Transfer Learning by Means of Ensemble Learning and Swarm Intelligence-Based Neuroevolution
Abstract
Neural Architecture Search (NAS) methods, when applied to very small but complex datasets, tend to overfit on the validation partitions and underperform compared to Transfer Learning models. In order to reduce the bias and variance of their predictions, Deep Ensemble Learning (DEL) can be used. The combination of NAS and DEL has only been employed on large datasets in the literature, but these scenarios do not present the overfitting in validation we typically experience, for instance, on medical imaging applications. In this work, we empirically assess the feasibility of NAS, DEL and the combination of the two on both small and large dataset scenarios. We find that the performance of the ensembles highly depend on the degree of overfitting of the standalone models, but always will outperform the worst generalizing models in the population.
Cite
Text
Gómez et al. "Improving Transfer Learning by Means of Ensemble Learning and Swarm Intelligence-Based Neuroevolution." Proceedings of the Third International Conference on Automated Machine Learning, 2024.Markdown
[Gómez et al. "Improving Transfer Learning by Means of Ensemble Learning and Swarm Intelligence-Based Neuroevolution." Proceedings of the Third International Conference on Automated Machine Learning, 2024.](https://mlanthology.org/automl/2024/gomez2024automl-improving/)BibTeX
@inproceedings{gomez2024automl-improving,
title = {{Improving Transfer Learning by Means of Ensemble Learning and Swarm Intelligence-Based Neuroevolution}},
author = {Gómez, Adri and Abella, Monica and Desco, Manuel},
booktitle = {Proceedings of the Third International Conference on Automated Machine Learning},
year = {2024},
pages = {7/1-25},
volume = {256},
url = {https://mlanthology.org/automl/2024/gomez2024automl-improving/}
}