A User Comfort Model and Index Policy for Personalizing Discrete Controller Decisions
OPEN ACCESS
Loading...
Author / Producer
Date
2018-11-29
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
User feedback allows for tailoring system operation to ensure individual user satisfaction. A major challenge in personalized decision-making is the systematic construction of a user model during operation while maintaining control performance. This paper presents both an index-based control policy to smartly collect and process user feedback and a user comfort model in the form of a Markov decision process with a priori unknown user-specific state transition probabilities. The control policy utilizes explicit user feedback to optimize a reward measure reflecting user comfort and addresses the exploration-exploitation trade-off in a multi-armed bandit framework. The proposed approach combines restless bandits and upper confidence bound algorithms. It introduces an exploration term into the restless bandit formulation, utilizes user feedback to identify the user model, and is shown to be indexable. We demonstrate its capabilities with a simulation for learning a user’s trade-off between comfort and energy usage.
Permanent link
Publication status
published
External links
Editor
Book title
2018 European Control Conference (ECC)
Journal / series
Volume
Pages / Article No.
1759 - 1765
Publisher
IEEE
Event
16th European Control Conference (ECC 2018)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Organisational unit
09563 - Zeilinger, Melanie / Zeilinger, Melanie
Notes
Funding
157601 - Safety and Performance for Human in the Loop Control (SNF)
Related publications and datasets
Is cited by: 10.3929/ethz-b-000449554