
Open access
Date
2020Type
- Conference Paper
Abstract
We introduce a stochastic contextual bandit model where at each time step the environment chooses a distribution over a context set and samples the context from this distribution. The learner observes only the context distribution while the exact context realization remains hidden. This allows for a broad range of applications where the context is stochastic or when the learner needs to predict the context. We leverage the UCB algorithm to this setting and show that it achieves an order-optimal high-probability bound on the cumulative regret for linear and kernelized reward functions. Our results strictly generalize previous work in the sense that both our model and the algorithm reduce to the standard setting when the environment chooses only Dirac delta distributions and therefore provides the exact context to the learner. We further analyze a variant where the learner observes the realized context after choosing the action. Finally, we demonstrate the proposed method on synthetic and real-world datasets. Show more
Permanent link
https://doi.org/10.3929/ethz-b-000385952Publication status
publishedEditor
Book title
Advances in Neural Information Processing Systems 32Volume
Pages / Article No.
Publisher
CurranEvent
Organisational unit
03908 - Krause, Andreas / Krause, Andreas
Funding
159557 - Explore-exploit with Gaussian Processes under Complex Constraints (SNF)
815943 - Reliable Data-Driven Decision Making in Cyber-Physical Systems (EC)
Notes
Poster presented on December 12, 2019More
Show all metadata