Using Relative Novelty to Identify Useful Temporal Abstractions in Reinforcement Learning

Simsek, Özgür; Barto, Andrew G.

doi:10.1145/1015330.1015353

Using Relative Novelty to Identify Useful Temporal Abstractions in Reinforcement Learning

Özgür Simsek, Andrew G. Barto

ICML 2004

doi:10.1145/1015330.1015353 /icml/2004/simsek2004icml-using/

Abstract

We present a new method for automatically creating useful temporal abstractions in reinforcement learning. We argue that states that allow the agent to transition to a different region of the state space are useful subgoals, and propose a method for identifying them using the concept of relative novelty. When such a state is identified, a temporally-extended activity (e.g., an option) is generated that takes the agent efficiently to this state. We illustrate the utility of the method in a number of tasks.

PDF ICML Semantic Scholar

Cite

Text

Simsek and Barto. "Using Relative Novelty to Identify Useful Temporal Abstractions in Reinforcement Learning." International Conference on Machine Learning, 2004. doi:10.1145/1015330.1015353

Markdown

[Simsek and Barto. "Using Relative Novelty to Identify Useful Temporal Abstractions in Reinforcement Learning." International Conference on Machine Learning, 2004.](https://mlanthology.org/icml/2004/simsek2004icml-using/) doi:10.1145/1015330.1015353

BibTeX

@inproceedings{simsek2004icml-using,
  title     = {{Using Relative Novelty to Identify Useful Temporal Abstractions in Reinforcement Learning}},
  author    = {Simsek, Özgür and Barto, Andrew G.},
  booktitle = {International Conference on Machine Learning},
  year      = {2004},
  doi       = {10.1145/1015330.1015353},
  url       = {https://mlanthology.org/icml/2004/simsek2004icml-using/}
}