A User Comfort Model and Index Policy for Personalizing Discrete Controller Decisions
Zeilinger, Melanie N.
- Conference Paper
Rights / licenseIn Copyright - Non-Commercial Use Permitted
User feedback allows for tailoring system operation to ensure individual user satisfaction. A major challenge in personalized decision-making is the systematic construction of a user model during operation while maintaining control performance. This paper presents both an index-based control policy to smartly collect and process user feedback and a user comfort model in the form of a Markov decision process with a priori unknown user-specific state transition probabilities. The control policy utilizes explicit user feedback to optimize a reward measure reflecting user comfort and addresses the exploration-exploitation trade-off in a multi-armed bandit framework. The proposed approach combines restless bandits and upper confidence bound algorithms. It introduces an exploration term into the restless bandit formulation, utilizes user feedback to identify the user model, and is shown to be indexable. We demonstrate its capabilities with a simulation for learning a user’s trade-off between comfort and energy usage. Show more
Book title2018 European Control Conference (ECC)
Pages / Article No.
Organisational unit09563 - Zeilinger, Melanie / Zeilinger, Melanie
157601 - Safety and Performance for Human in the Loop Control (SNF)
MoreShow all metadata