Monte Carlo Tree Search in Continuous Spaces Using Voronoi Optimistic Optimization with Regret Bounds
Abstract
Many important applications, including robotics, data-center management, and process control, require planning action sequences in domains with continuous state and action spaces and discontinuous objective functions. Monte Carlo tree search (MCTS) is an effective strategy for planning in discrete action spaces. We provide a novel MCTS algorithm (voot) for deterministic environments with continuous action spaces, which, in turn, is based on a novel black-box function-optimization algorithm (voo) to efficiently sample actions. The voo algorithm uses Voronoi partitioning to guide sampling, and is particularly efficient in high-dimensional spaces. The voot algorithm has an instance of voo at each node in the tree. We provide regret bounds for both algorithms and demonstrate their empirical effectiveness in several high-dimensional problems including two difficult robotics planning problems.
Cite
Text
Kim et al. "Monte Carlo Tree Search in Continuous Spaces Using Voronoi Optimistic Optimization with Regret Bounds." AAAI Conference on Artificial Intelligence, 2020. doi:10.1609/AAAI.V34I06.6546Markdown
[Kim et al. "Monte Carlo Tree Search in Continuous Spaces Using Voronoi Optimistic Optimization with Regret Bounds." AAAI Conference on Artificial Intelligence, 2020.](https://mlanthology.org/aaai/2020/kim2020aaai-monte/) doi:10.1609/AAAI.V34I06.6546BibTeX
@inproceedings{kim2020aaai-monte,
title = {{Monte Carlo Tree Search in Continuous Spaces Using Voronoi Optimistic Optimization with Regret Bounds}},
author = {Kim, Beomjoon and Lee, Kyungjae and Lim, Sungbin and Kaelbling, Leslie Pack and Lozano-Pérez, Tomás},
booktitle = {AAAI Conference on Artificial Intelligence},
year = {2020},
pages = {9916-9924},
doi = {10.1609/AAAI.V34I06.6546},
url = {https://mlanthology.org/aaai/2020/kim2020aaai-monte/}
}