Boosting With the L2 Loss: Regression and Classification
METADATA ONLY
Loading...
Author / Producer
Date
2003
Publication Type
Journal Article
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
his article investigates a computationally simple variant of boosting, L2Boost, which is constructed from a functional gradient descent algorithm with the L2-loss function. Like other boosting algorithms, L2Boost uses many times in an iterative fashion a prechosen fitting method, called the learner. Based on the explicit expression of refitting of residuals of L2Boost, the case with (symmetric) linear learners is studied in detail in both regression and classification. In particular, with the boosting iteration m working as the smoothing or regularization parameter, a new exponential bias-variance trade-off is found with the variance (complexity) term increasing very slowly as m tends to infinity. When the learner is a smoothing spline, an optimal rate of convergence result holds for both regression and classification and the boosted smoothing spline even adapts to higher-order, unknown smoothness. Moreover, a simple expansion of a (smoothed) 0–1 loss function is derived to reveal the importance of the decision boundary, bias reduction, and impossibility of an additive bias-variance decomposition in classification. Finally, simulation and real dataset results are obtained to demonstrate the attractiveness of L2Boost. In particular, we demonstrate that L2Boosting with a novel component-wise cubic smoothing spline is both practical and effective in the presence of high-dimensional predictors.
Permanent link
Publication status
published
External links
Editor
Book title
Journal / series
Volume
98 (462)
Pages / Article No.
324 - 339
Publisher
Taylor & Francis
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Functional gradient descent; LogitBoost; Minimax error rate; Nonparametric classification; Nonparametric regression; Smoothing spline
Organisational unit
03502 - Bühlmann, Peter L. / Bühlmann, Peter L.