Stochasticity in Feedback Loops: Great Expectations and Guaranteed Ruin

Abstract

Stochastic components in a feedback loop introduce state behaviors that are fundamentally different from those observed in a deterministic system. The effect of injecting a stochastic signal additively in linear feedback systems can be viewed as the addition of filtered stochastic noise. If the stochastic signal enters the feedback loop in a multiplicative manner, a much richer set of state behaviors emerges. These phenomena are investigated for the simplest possible system; a multiplicative noise in a scalar, integrating feedback loop. The same dynamics arise when considering a first-order system in feedback with a stochastic gain. Dynamics of this form arise naturally in a number of domains, including compound investment in finance, chemical reaction dynamics, population dynamics, epidemiology, control over lossy communication channels, and adaptive control. Understanding the nature of such dynamics in a simple system is a precursor to recognizing them in more complex stochastic dynamical systems.