Graduate student in Atmospheric Sciences researching applications of machine learning to ensemble weather forecasting
Can machines learn to predict the weather? More specifically, can we train deep convolutional neural networks to forecast globally-gridded weather data solely from past observations, that is, with no knowledge of the physics of the atmosphere? We have tried exactly that, and found that deep learning can significantly outperform benchmarks such as climatology, persistence, and the physical barotropic vorticity model. While the neural networks don't perform as well as operational weather models, we still have a lot more atmospheric variables to add to the mix!
A fundamental problem of ensemble weather forecasting is statistical post-processing of ensemble data. How can we determine which of the ensemble solutions will be closest to the observed weather given that all solutions are nominally equally likely? We turn to deep learning to help answer this question, and examine its capability to produce meaningful, forecaster-ready weather information.
Deep convection and severe thunderstorms offer a unique environment for testing the Lorenzian theory of intrinsic weather predictability. For my Master's work, my advisor Dr. Dale Durran and I used idealized and real-environment weather simulations to make important demonstrations that small errors in model initial conditions on the large scales (e.g., synoptic fronts) have just as much influence as large errors on scales as small as butterflies.
DLWP is no ordinary weather forecasting model. It doesn't know physics and it doesn't calculate how atmospheric motions evolve. Instead, it learns to emulate this solely by recognizing patterns in historical weather data. The code repository for DLWP includes modules to download and process the reanalysis data, long with robust code for building neural networks in Keras and TensorFlow and using them to forecast time series of weather.
MOS-X is a simple proof-of-concept machine-learning based weather prediction model built using scikit-learn as an improvement over traditional Model Output Statistics (MOS). State-of-the-art numerical weather prediction (NWP) models, such as the Global Forecast System (GFS) and North American Mesoscale (NAM) models, do not represent localized weather well, and therefore rely on MOS to produce forecasts of surface parameters. MOS is an old, multiple-linear-regression technique that has mixed success, so I improved it using random forests and gradient boosting trees. It is currently being entered as a model in the WxChallenge collegiate weather forecasting competition and competes well with human forecasters! In fact, it even performed better than all human forecasters at Pueblo, CO in 2019.
Since October 2015 I have managed a personal weather station reporting weather at the University of Washington's Atmospheric Sciences-Geophysics building. The data are displayed on my weather station's website here.
With so many different sources of weather forecasts out there (National Weather Service, Weather Channel, AccuWeather, to name a few) how do we know which one is most accurate? And, supposing you're an amateur weather forecaster, how do you put all that data together into a better forecast for tomorrow's weather? That's what theta-e is built for.
To address my second topic of research above, I am developing code to analyze weather model data from the NCAR operational ensemble forecasts and calculate verification metrics from observed radar and surface data. This work-in-progress will ultimately be a generalized tool to build AI models to predict many weather parameters from these datasets.
Weyn, J. A., Durran, D. R., & Caruana, R. (2019). Can machines learn to predict weather? Using deep learning to predict gridded 500-hPa geopotential height from historical weather data. JAMES, 11. DOI
Weyn, J. A., & Durran, D. R. (2018). The scale dependence of initial-condition sensitivities in simulations of convective systems over the Southeastern US. Quarterly Journal of the Royal Meteorological Society. DOI
Weyn, J. A., & Durran, D. R. (2018). Ensemble spread grows more rapidly in higher-resolution simulations of deep convection. Journal of the Atmospheric Sciences, 75(10), 3331-3345. DOI
Durran, D., Weyn, J. A., & Menchaca, M. Q. (2017). Practical Considerations for Computing Dimensional Spectra from Gridded Data. Monthly Weather Review, 145(9), 3901–3910. DOI
Weyn, J. A., & Durran, D. R. (2017). The dependence of the predictability of mesoscale convective systems on the horizontal scale and amplitude of initial errors in idealized simulations. Journal of the Atmospheric Sciences, 74(7), 2191–2210. DOI
Durran, D. R., & Weyn, J. A. (2016). Thunderstorms Do Not Get Butterflies. Bulletin of the American Meteorological Society, 97(2), 237–243. DOI