A Resource-Efficient Method for Repeated HPO and NAS Problems
Abstract
In this work we consider the problem of repeated hyperparameter and neural architecture search (HNAS).We propose an extension of Successive Halving that is able to leverage information gained in previous HNAS problems with the goal of saving computational resources. We empirically demonstrate that our solution is able to drastically decrease costs while maintaining accuracy and being robust to negative transfer. Our method is significantly simpler than competing transfer learning approaches, setting a new baseline for transfer learning in HNAS.
Cite
Text
Zappella et al. "A Resource-Efficient Method for Repeated HPO and NAS Problems." ICML 2021 Workshops: AutoML, 2021.Markdown
[Zappella et al. "A Resource-Efficient Method for Repeated HPO and NAS Problems." ICML 2021 Workshops: AutoML, 2021.](https://mlanthology.org/icmlw/2021/zappella2021icmlw-resourceefficient/)BibTeX
@inproceedings{zappella2021icmlw-resourceefficient,
title = {{A Resource-Efficient Method for Repeated HPO and NAS Problems}},
author = {Zappella, Giovanni and Salinas, David and Archambeau, Cedric},
booktitle = {ICML 2021 Workshops: AutoML},
year = {2021},
url = {https://mlanthology.org/icmlw/2021/zappella2021icmlw-resourceefficient/}
}