Stochastic Bandits with Context Distributions
OPEN ACCESS
Loading...
Author / Producer
Date
2020
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
We introduce a stochastic contextual bandit model where at each time step the environment chooses a distribution over a context set and samples the context from this distribution. The learner observes only the context distribution while the exact context realization remains hidden. This allows for a broad range of applications where the context is stochastic or when the learner needs to predict the context. We leverage the UCB algorithm to this setting and show that it achieves an order-optimal high-probability bound on the cumulative regret for linear and kernelized reward functions. Our results strictly generalize previous work in the sense that both our model and the algorithm reduce to the standard setting when the environment chooses only Dirac delta distributions and therefore provides the exact context to the learner. We further analyze a variant where the learner observes the realized context after choosing the action. Finally, we demonstrate the proposed method on synthetic and real-world datasets.
Permanent link
Publication status
published
Book title
Advances in Neural Information Processing Systems 32
Journal / series
Volume
18
Pages / Article No.
14046 - 14055
Publisher
Curran
Event
33rd Conference on Neural Information Processing Systems (NeurIPS 2019)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Organisational unit
03908 - Krause, Andreas / Krause, Andreas
Notes
Poster presented on December 12, 2019
Funding
159557 - Explore-exploit with Gaussian Processes under Complex Constraints (SNF)
815943 - Reliable Data-Driven Decision Making in Cyber-Physical Systems (EC)
815943 - Reliable Data-Driven Decision Making in Cyber-Physical Systems (EC)