A Strategy for Ranking Optimization Methods Using Multiple Criteria
Abstract
Many methods for optimizing black-box functions exist, and many metrics exist for judging the performance of a specific optimization method. There is not, however, a generally agreed upon strategy for simultaneously comparing the performance of multiple optimization methods for multiple performance metrics across a range of optimization problems. This paper proposes such a methodology, which uses nonparametric statistical tests to convert the metrics recorded for each problem into a partial ranking of optimization methods; these partial rankings are then amalgamated through a voting mechanism to generate a final score for each optimization method. Mathematical analysis is provided to motivate decisions within this strategy, and numerical results are provided to demonstrate the potential insights afforded thereby.
Cite
Text
Dewancker et al. "A Strategy for Ranking Optimization Methods Using Multiple Criteria." Proceedings of the Workshop on Automatic Machine Learning, 2016.Markdown
[Dewancker et al. "A Strategy for Ranking Optimization Methods Using Multiple Criteria." Proceedings of the Workshop on Automatic Machine Learning, 2016.](https://mlanthology.org/automl/2016/dewancker2016automl-strategy/)BibTeX
@inproceedings{dewancker2016automl-strategy,
title = {{A Strategy for Ranking Optimization Methods Using Multiple Criteria}},
author = {Dewancker, Ian and McCourt, Michael and Clark, Scott and Hayes, Patrick and Johnson, Alexandra and Ke, George},
booktitle = {Proceedings of the Workshop on Automatic Machine Learning},
year = {2016},
pages = {11-20},
volume = {64},
url = {https://mlanthology.org/automl/2016/dewancker2016automl-strategy/}
}