Discovering User Types: Characterization of User Traits by Task-Specific Behaviors in Reinforcement Learning
Abstract
We often want to infer user traits when personalizing interventions. Approaches like Inverse RL can learn traits formalized as parameters of a Markov Decision Process but are data intensive. Instead of inferring traits for individuals, we study the relationship between RL worlds and the set of user traits. We argue that understanding the breakdown of ``user types" within a world -- broad sets of traits that result in the same behavior -- helps rapidly personalize interventions. We show that seemingly different RL worlds admit the same set of user types and formalize this observation as an equivalence relation defined on worlds. We show that these equivalence classes capture many different worlds. We argue that the richness of these classes allows us to transfer insights on intervention design between toy and real worlds.
Cite
Text
Ankile et al. "Discovering User Types: Characterization of User Traits by Task-Specific Behaviors in Reinforcement Learning." ICML 2023 Workshops: ILHF, 2023.Markdown
[Ankile et al. "Discovering User Types: Characterization of User Traits by Task-Specific Behaviors in Reinforcement Learning." ICML 2023 Workshops: ILHF, 2023.](https://mlanthology.org/icmlw/2023/ankile2023icmlw-discovering/)BibTeX
@inproceedings{ankile2023icmlw-discovering,
title = {{Discovering User Types: Characterization of User Traits by Task-Specific Behaviors in Reinforcement Learning}},
author = {Ankile, Lars Lien and Ham, Brian and Mao, Kevin and Shin, Eura and Swaroop, Siddharth and Doshi-Velez, Finale and Pan, Weiwei},
booktitle = {ICML 2023 Workshops: ILHF},
year = {2023},
url = {https://mlanthology.org/icmlw/2023/ankile2023icmlw-discovering/}
}