Nonconvex Meta-Optimization for Deep Learning
Abstract
Hyperparameter tuning in mathematical optimization is a notoriously difficult problem. Recent tools from online control give rise to a provable methodology for hyperparameter tuning in convex optimization called meta-optimization. In this work, we extend this methodology to nonconvex optimization and the training of deep neural networks. We present an algorithm for nonconvex meta-optimization that leverages the reduction from nonconvex optimization to convex optimization, and investigate its applicability for deep learning tasks on academic-scale datasets.
Cite
Text
Chen et al. "Nonconvex Meta-Optimization for Deep Learning." ICML 2024 Workshops: HiLD, 2024.Markdown
[Chen et al. "Nonconvex Meta-Optimization for Deep Learning." ICML 2024 Workshops: HiLD, 2024.](https://mlanthology.org/icmlw/2024/chen2024icmlw-nonconvex/)BibTeX
@inproceedings{chen2024icmlw-nonconvex,
title = {{Nonconvex Meta-Optimization for Deep Learning}},
author = {Chen, Xinyi and Dogariu, Evan and Lu, Zhou and Hazan, Elad},
booktitle = {ICML 2024 Workshops: HiLD},
year = {2024},
url = {https://mlanthology.org/icmlw/2024/chen2024icmlw-nonconvex/}
}