Iterative Model Refinement of Recommender MDPs Based on Expert Feedback
Abstract
In this paper, we present a method to iteratively refine the parameters of a Markov Decision Process by leveraging constraints implied from an expert’s review of the policy. We impose a constraint on the parameters of the model for every case where the expert’s recommendation differs from the recommendation of the policy. We demonstrate that consistency with an expert’s feedback leads to non-convex constraints on the model parameters. We refine the parameters of the model, under these constraints, by partitioning the parameter space and iteratively applying alternating optimization. We demonstrate how the approach can be applied to both flat and factored MDPs and present results based on diagnostic sessions from a manufacturing scenario.
Cite
Text
Khan et al. "Iterative Model Refinement of Recommender MDPs Based on Expert Feedback." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2013. doi:10.1007/978-3-642-40988-2_11Markdown
[Khan et al. "Iterative Model Refinement of Recommender MDPs Based on Expert Feedback." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2013.](https://mlanthology.org/ecmlpkdd/2013/khan2013ecmlpkdd-iterative/) doi:10.1007/978-3-642-40988-2_11BibTeX
@inproceedings{khan2013ecmlpkdd-iterative,
title = {{Iterative Model Refinement of Recommender MDPs Based on Expert Feedback}},
author = {Khan, Omar Zia and Poupart, Pascal and Agosta, John Mark},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2013},
pages = {162-177},
doi = {10.1007/978-3-642-40988-2_11},
url = {https://mlanthology.org/ecmlpkdd/2013/khan2013ecmlpkdd-iterative/}
}