Delayed deep learning for continuous-time dynamical systems

Abstract

Bridging the gap between deep learning and dynamical systems, neural ODEs are a promising approach to model continuous-time dynamical systems. Motivated by state augmentation in discrete-time models, we propose to extend the neural ODE framework to neural delay di erential equations in order to naturally capture non-Markovian e ects such as time delays or hysteresis, which are often encountered in real world applications. We demonstrate the superior performance of neural delay di erential equations on the task of modelling a partially observed oscillator in comparison with augmented neural ODEs. Moreover, we showcase robustness to observation noise, generalization over time and initial conditions, and the expressive power on more complex dynamical systems. Furthermore, a result on universal approximation is provided and the connection to delay embeddings is discussed. In an exploratory part, we discuss deep learning approaches for stability analysis of time delay systems and propose to jointly learn a dynamics model and a Lyapunov-Razumikhin function via discretization of the Razumikhin condition. The applicability of this approach is demonstrated for the task of stabilizing an inverted pendulum with delayed feedback control.