Subseasonal Climate Forecasting

Improving our ability to forecast the weather and climate is of interest to all sectors of the economy and to government agencies from the local to the national level. Weather forecasts zero to ten days ahead and climate forecasts seasons to decades ahead are currently used operationally in decision-making, and the accuracy and reliability of these forecasts has improved consistently in recent decades (Troccoli, 2010). However, many critical applications – including water allocation, wildfire management, and drought and flood mitigation – require subseasonal forecasts with lead times in between these two extremes (Merryfield et al., 2020; White et al., 2017).

While short-term forecasting accuracy is largely sustained by physics-based dynamical models, these deterministic methods have limited subseasonal accuracy due to chaos (Lorenz, 1963). Indeed, subseasonal forecasting has long been considered a “predictability desert” due to its complex dependence on both local weather and global climate variables (Vitart et al., 2012). Recent studies (opens in new tab), however, have highlighted important sources of predictability on subseasonal timescales, and the focus of several recent large-scale research efforts has been to advance the subseasonal capabilities of operational physics-based models (Vitart et al., 2017; Pegion et al., 2019; Lang et al., 2020). Our team has undertaken a parallel effort to demonstrate the value of machine learning methods in improving subseasonal forecasting.

Our team has undertaken a parallel effort to demonstrate the value of machine learning methods in improving subseasonal forecasting.

The Subseasonal Climate Forecast Rodeo

To improve the accuracy of subseasonal forecasts, the U.S. Bureau of Reclamation (USBR) and the National Oceanic and Atmospheric Administration (NOAA) launched the Subseasonal Climate Forecast Rodeo (opens in new tab), a year-long real-time forecasting challenge in which participants aimed to skillfully predict temperature and precipitation in the western U.S. two to four weeks and four to six weeks in advance. Our team developed a machine learning approach to the Rodeo and a SubseasonalRodeo dataset (opens in new tab) for training and evaluating subseasonal forecasting systems.

subseasonal climate forecast charts of the United States — Example submission for the week 3-4 temperature forecast targeting the two-week period starting on February 5th, 2018. Our submission is on the upper left, while the upper right shows the realized temperature anomalies. The bottom left is the forecast of the US operational dynamical model Climate Forecasting System v2. Damped persistence is a standard meteorological forecasting method used as a Rodeo baseline.

Our final Rodeo solution was an ensemble of two nonlinear regression models. The first integrates a diverse collection of meteorological measurements and dynamic model forecasts and prunes irrelevant predictors using a customized multitask model selection procedure. The second uses only historical measurements of the target variable (temperature or precipitation) and introduces multitask nearest neighbor features into a weighted local linear regression. Each model alone outperforms the debiased operational U.S. Climate Forecasting System (CFSv2), and, over 2011-2018, an ensemble of our regression models and debiased CFSv2 improves debiased CFSv2 skill by 40-50% for temperature and 129-169% for precipitation. See our write-up Improving Subseasonal Forecasting in the Western U.S. with Machine Learning (opens in new tab) for more details. While this work demonstrated the promise of machine learning models for subseasonal forecasting, it also highlighted the complementary strengths of physics- and learning-based approaches and the opportunity to combine those strengths to improve forecasting skill.

Adaptive Bias Correction (ABC)

To harness the complementary strengths of physics- and learning-based models, we next developed a hybrid dynamical-learning framework for improved subseasonal forecasting. In particular, we learn to adaptively correct the biases of dynamical models and apply our novel adaptive bias correction (ABC) to improve the skill of subseasonal temperature and precipitation forecasts.

Spatial distribution of precipitation skill

Spatial distribution of temperature skill

ABC is an ensemble of three new low-cost, high-accuracy machine learning models: Dynamical++, Climatology++, and Persistence++. Each model trains only on past temperature, precipitation, and forecast data and outputs corrections for future forecasts tailored to the site, target date, and dynamical model. Dynamical++ and Climatology++ learn site- and date-specific offsets for dynamical and climatological forecasts by minimizing forecasting error over adaptively-selected training periods. Persistence++ additionally accounts for recent weather trends by combining lagged observations, dynamical forecasts, and climatology to minimize historical forecasting error for each site.

ABC can be applied operationally as a computationally inexpensive enhancement to any dynamical model forecast, and we use this property to substantially reduce the forecasting errors of eight operational dynamical models, including the state-of-the-art ECMWF model.

We couple these performance improvements with a practical workflow for explaining ABC skill gains using Cohort Shapley (Mase et al., 2019) and identifying higher-skill windows of opportunity (Mariotti et al., 2020) based on relevant climate variables.

impact of hgt_500_pc1 on ABC skill improvement

forecast with largest hgt_500_pc1 impact

To facilitate future deployment and development, we also release our model and workflow code through the subseasonal_toolkit (opens in new tab) Python package. See our write-up Adaptive Bias Correction for Improved Subseasonal Forecasting (opens in new tab) for more details.

The SubseasonalClimateUSA Dataset

To train and evaluate our contiguous US models, we developed a SubseasonalClimateUSA dataset (opens in new tab) housing a diverse collection of ground-truth measurements and model forecasts relevant to subseasonal timescales. The SubseasonalClimateUSA dataset is updated regularly and publicly accessible via the subseasonal_data (opens in new tab) package. In SubseasonalClimateUSA: A Dataset for Subseasonal Forecasting and Benchmarking (opens in new tab), we used this dataset to benchmark ABC against operational dynamical models and seven state-of-the-art deep learning and machine learning methods from the literature. In each subseasonal forecasting task (predicting precipitation or temperature in weeks 3-4 or weeks 5-6), ABC and its component models provided the best performance.

subseasonal forecasting precipitation graphs — Percentage improvement over mean debiased CFSv2 RMSE when forecasting precipitation in the contiguous U.S. over 2011-2020. White grid points indicate negative or 0% improvement

Online Learning with Optimism and Delay

To provide more flexible and adaptive model ensembling while respecting the demands of real-time climate and weather forecasting, we developed three new optimistic online learning algorithms — AdaHedgeD, DORM, and DORM+ — that require no parameter tuning and have optimal regret guarantees under delayed feedback.

online learning regret plot — Each year, the PoolD (opens in new tab) online learning algorithms exhibit small regret relative to the best input model in hindsight, despite observing only 26 observations per year.

Our open-source Python implementation, available via the PoolD (opens in new tab) library, provides simple strategies for combining the forecasts of different subseasonal forecasting models, adapting the weights of each model based on real-time performance. See our write-up Online Learning with Optimism and Delay (opens in new tab) for more details.

Looking forward

We’re excited to continue to explore machine learning applied to subseasonal forecasting on a global scale, and we hope that our open-source packages will facilitate future subseasonal development and benchmarking. If you have ideas for model or dataset development, please contribute to our open-source Python code or reach out to Lester Mackey!