Logistic regression with total variation regularization

Abstract

We study logistic regression with total variation penalty on the canonical parameter andshow that the resulting estimator satisfies a sharp oracle inequality: the excess risk of the estimator isadaptive to the number of jumps of the underlying signal or an approximation thereof. In particular,when there are finitely many jumps, and jumps up are sufficiently separated from jumps down, thenthe estimator converges with a parametric rate up to a logarithmic term logn/n, provided the tuningparameter is chosen appropriately of order 1/√n. Our results extend earlier results for quadraticloss to logistic loss. We do not assume any a priori known bounds on the canonical parameter, butinstead only make use of the local curvature of the theoretical risk.