Uncertainty Quantification for Machine Learning-Based Ionosphere and Space Weather Forecasting: Ensemble, Bayesian Neural Network, and Quantile Gradient Boosting
OPEN ACCESS
Author / Producer
Date
2023-10
Publication Type
Journal Article
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
Machine learning (ML) has been increasingly applied to space weather and ionosphere problems in recent years, with the goal of improving modeling and forecasting capabilities through a data-driven modeling approach of nonlinear relationships. However, little work has been done to quantify the uncertainty of the results, lacking an indication of how confident and reliable the results of an ML system are. In this paper, we implement and analyze several uncertainty quantification approaches for an ML-based model to forecast Vertical Total Electron Content (VTEC) 1-day ahead and corresponding uncertainties with 95% confidence intervals (CI): (a) Super-Ensemble of ML-based VTEC models (SE), (b) Gradient Tree Boosting with quantile loss function (Quantile Gradient Boosting, QGB), (c) Bayesian neural network (BNN), and (d) BNN including data uncertainty (BNN + D). Techniques that consider only model parameter uncertainties (a and c) predict narrow CI and over-optimistic results, whereas accounting for both model parameter and data uncertainties with the BNN + D approach leads to a wider CI and the most realistic uncertainties quantification of VTEC forecast. However, the BNN + D approach suffers from a high computational burden, while the QGB approach is the most computationally efficient solution with slightly less realistic uncertainties. The QGB CI are determined to a large extent from space weather indices, as revealed by the feature analysis. They exhibit variations related to daytime/nightime, solar irradiance, geomagnetic activity, and post-sunset low-latitude ionosphere enhancement.
Permanent link
Publication status
published
External links
Editor
Book title
Journal / series
Volume
21 (10)
Pages / Article No.
Publisher
Wiley
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
machine learning; uncertainty quantification; confidence intervals; probabilistic ionosphere forecast; space weather; ensemble; Bayesian neural network; quantile gradient boosting
Organisational unit
09707 - Soja, Benedikt / Soja, Benedikt