Regularized System Identification: A Hierarchical Bayesian Approach

Abstract

In this paper, the hierarchical Bayesian method for regularized system identification is introduced. To this end, a hyperprior distribution is considered for the regularization matrix and then, the impulse response and the regularization matrix are jointly estimated based on a maximum a posteriori (MAP) approach. Toward introducing a suitable hyperprior, we decompose the regularization matrix using Cholesky decomposition and reduce the estimation problem to the cone of upper triangular matrices with positive diagonal entries. Following this, the hyperprior is introduced on a designed sub-cone of this set. The method differs from the current trend in regularized system identification from various aspect, e.g., the estimation is performed by solving a single stage problem. The MAP estimation problem reduces to a multi-convex optimization problem and a sequential convex programming algorithm is introduced for solving this problem. Consequently, the proposed method is a computationally efficient strategy specially when the regularization matrix has a large size. The method is numerically verified on benchmark examples. Owing to the employed full Bayesian approach, the estimation method shows a satisfactory bias-variance trade-off.