A Unified Switching System Perspective and Convergence Analysis of Q-Learning Algorithms
- Conference Paper
This paper develops a novel and unified framework to analyze the convergence of a large family of Q-learning algorithms from the switching system perspective. We show that the nonlinear ODE models associated with Q-learning and many of its variants can be naturally formulated as affine switching systems. Building on their asymptotic stability, we obtain a number of interesting results: (i) we provide a simple ODE analysis for the convergence of asynchronous Q-learning under relatively weak assumptions; (ii) we establish the first convergence analysis of the averaging Q-learning algorithm; and (iii) we derive a new sufficient condition for the convergence of Q-learning with linear function approximation. Show more
Book titleAdvances in Neural Information Processing Systems 33
Pages / Article No.
Organisational unit09729 - He, Niao / He, Niao
NotesDue to the Coronavirus (COVID-19) the conference was conducted virtually.
MoreShow all metadata