Long-Run Behaviour of Multi-Fidelity Bayesian Optimisation

Abstract

Multi-fidelity Bayesian Optimisation (MFBO) has been shown to generally converge faster than single-fidelity Bayesian Optimisation (SFBO) (\cite{poloczek2017multi}). Inspired by recent benchmark papers, we are investigating the long-run behaviour of MFBO, based on observations in the literature that it might under-perform in certain scenarios (\cite{mikkola2023multi}, \cite{eggensperger2021hpobench}). An under-performance of MBFO in the long-run could significantly undermine its application to many research tasks, especially when we are not able to identify when the under-performance begins, and other BO algorithms would have performed better. We create a simple benchmark study, showcase empirical results and discuss scenarios, concluding with inconclusive results.