Unobserved Is Not Equal to Non-Existent: Using Gaussian Processes to Infer Immediate Rewards Across Contexts

Hamoon Azizsoltani, Yeo-Jin Kim, Markel Sanz Ausin, Tiffany Barnes, Min Chi

IJCAI 2019 pp. 1974-1980

doi:10.24963/IJCAI.2019/273 /ijcai/2019/azizsoltani2019ijcai-unobserved/

Abstract

Learning optimal policies in real-world domains with delayed rewards is a major challenge in Reinforcement Learning. We address the credit assignment problem by proposing a Gaussian Process (GP)-based immediate reward approximation algorithm and evaluate its effectiveness in 4 contexts where rewards can be delayed for long trajectories. In one GridWorld game and 8 Atari games, where immediate rewards are available, our results showed that on 7 out 9 games, the proposed GP-inferred reward policy performed at least as well as the immediate reward policy and significantly outperformed the corresponding delayed reward policy. In e-learning and healthcare applications, we combined GP-inferred immediate rewards with offline Deep Q-Network (DQN) policy induction and showed that the GP-inferred reward policies outperformed the policies induced using delayed rewards in both real-world contexts.

PDF IJCAI Semantic Scholar

Cite

Text

Azizsoltani et al. "Unobserved Is Not Equal to Non-Existent: Using Gaussian Processes to Infer Immediate Rewards Across Contexts." International Joint Conference on Artificial Intelligence, 2019. doi:10.24963/IJCAI.2019/273

Markdown

[Azizsoltani et al. "Unobserved Is Not Equal to Non-Existent: Using Gaussian Processes to Infer Immediate Rewards Across Contexts." International Joint Conference on Artificial Intelligence, 2019.](https://mlanthology.org/ijcai/2019/azizsoltani2019ijcai-unobserved/) doi:10.24963/IJCAI.2019/273

BibTeX

@inproceedings{azizsoltani2019ijcai-unobserved,
  title     = {{Unobserved Is Not Equal to Non-Existent: Using Gaussian Processes to Infer Immediate Rewards Across Contexts}},
  author    = {Azizsoltani, Hamoon and Kim, Yeo-Jin and Ausin, Markel Sanz and Barnes, Tiffany and Chi, Min},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year      = {2019},
  pages     = {1974-1980},
  doi       = {10.24963/IJCAI.2019/273},
  url       = {https://mlanthology.org/ijcai/2019/azizsoltani2019ijcai-unobserved/}
}