Recessions as Breadwinner for Forecasters State-Dependent Evaluation of Predictive Ability: Evidence from Big Macroeconomic US Data
OPEN ACCESS
Loading...
Author / Producer
Date
2019-10
Publication Type
Working Paper
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
This article re-examines the findings of Stock and Watson (2012b) who assessed the predictive performance of dynamic factor models (DFM) over autoregressive (AR) bench-marks for hundreds of target variables by focusing on possible business cycle performance asymmetries in the spirit of Chauvet and Potter (2013) and Siliverstovs (2017a, 2017b, 2019). Our forecasting experiment is based on a novel big macroeconomic dataset (FRED-QD) comprising over 200 quarterly indicators for almost 60 years (1960-2018; cf. e.g. McCracken & Ng, 2019b). Our results are consistent with this nascent state-dependent evaluation literature and generalize their relevance to a large number of indicators: We document systematic model performance differences across business cycles (longitudinal) as well as variable groups (cross-sectional). While the absolute size of prediction errors tend to be larger in busts than in booms for both DFMs and Arts, DFMs relative im-provement over Arts is typically large and statistically significant during recessions but not during expansions (cf. e.g. Chauvet & Potter, 2013). Our findings further suggest that the widespread practice of relying on full sample forecast evaluation metrics may not be ideal: For at least two thirds of all 216 macroeconomic indicators full sample relative RMSFEs systematically over-estimate performance in expansionary subsamples and under-estimate it in recessionary subsamples (cf. e.g. Siliverstovs, 2017a, 2019). These findings are robust to several alternative specifications and have high practical relevance for both consumers and producers of model-based economic forecasts.
Permanent link
Publication status
published
External links
Editor
Book title
Journal / series
Volume
463
Pages / Article No.
Publisher
KOF Swiss Economic Institute, ETH Zurich
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Forecast Evaluation; Dynamic factor models; Business Cycle Asymmetries; Big Macroeconomic Datasets; United States
Organisational unit
02525 - KOF Konjunkturforschungsstelle / KOF Swiss Economic Institute
06330 - KOF FB Konjunktur / KOF Macroeconomic forecasting