Discovering and Exploiting Additive Structure for Bayesian Optimization
Abstract
Bayesian optimization has proven invaluable for black-box optimization of expensive functions. Its main limitation is its exponential complexity with respect to the dimensionality of the search space using typical kernels. Luckily, many objective functions can be decomposed into additive subproblems, which can be optimized independently. We investigate how to automatically discover such (typically unknown) additive structure while simultaneously exploiting it through Bayesian optimization. We propose an efficient algorithm based on Metropolis-Hastings sampling and demonstrate its efficacy empirically on synthetic and real-world data sets. Throughout all our experiments we reliably discover hidden additive structure whenever it exists and exploit it to yield significantly faster convergence.
Cite
Text
Gardner et al. "Discovering and Exploiting Additive Structure for Bayesian Optimization." International Conference on Artificial Intelligence and Statistics, 2017.Markdown
[Gardner et al. "Discovering and Exploiting Additive Structure for Bayesian Optimization." International Conference on Artificial Intelligence and Statistics, 2017.](https://mlanthology.org/aistats/2017/gardner2017aistats-discovering/)BibTeX
@inproceedings{gardner2017aistats-discovering,
title = {{Discovering and Exploiting Additive Structure for Bayesian Optimization}},
author = {Gardner, Jacob R. and Guo, Chuan and Weinberger, Kilian Q. and Garnett, Roman and Grosse, Roger B.},
booktitle = {International Conference on Artificial Intelligence and Statistics},
year = {2017},
pages = {1311-1319},
url = {https://mlanthology.org/aistats/2017/gardner2017aistats-discovering/}
}