Properly Acting Under Partial Observability with Action Feasibility Constraints
Abstract
We introduce Action-Constrained Partially Observable Markov Decision Process (AC-POMDP), which arose from studying critical robotic applications with damaging actions. AC-POMDPs restrict the optimized policy to only apply feasible actions: each action is feasible in a subset of the state space, and the agent can observe the set of applicable actions in the current hidden state, in addition to standard observations. We present optimality equations for AC-POMDPs, which imply to operate on α -vectors defined over many different belief subspaces. We propose an algorithm named PreCondition Value Iteration (PCVI), which fully exploits this specific property of AC-POMDPs about α -vectors. We also designed a relaxed version of PCVI whose complexity is exponentially smaller than PCVI. Experimental results on POMDP robotic benchmarks with action feasibility constraints exhibit the benefits of explicitly exploiting the semantic richness of action-feasibility observations in AC-POMDPs over equivalent but unstructured POMDPs.
Cite
Text
Chanel and Teichteil-Königsbuch. "Properly Acting Under Partial Observability with Action Feasibility Constraints." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2013. doi:10.1007/978-3-642-40988-2_10Markdown
[Chanel and Teichteil-Königsbuch. "Properly Acting Under Partial Observability with Action Feasibility Constraints." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2013.](https://mlanthology.org/ecmlpkdd/2013/chanel2013ecmlpkdd-properly/) doi:10.1007/978-3-642-40988-2_10BibTeX
@inproceedings{chanel2013ecmlpkdd-properly,
title = {{Properly Acting Under Partial Observability with Action Feasibility Constraints}},
author = {Chanel, Caroline Ponzoni Carvalho and Teichteil-Königsbuch, Florent},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2013},
pages = {145-161},
doi = {10.1007/978-3-642-40988-2_10},
url = {https://mlanthology.org/ecmlpkdd/2013/chanel2013ecmlpkdd-properly/}
}