publications

2024

1. NeurIPS 2024

   

   Probabilistic predictions with Fourier neural operators

   Christopher Bülte, Scholl Philipp, and Kutyniok Gitta

   NeurIPS 2024 workshop on Bayesian Decision-making and Uncertainty, 2024

   Abs Bib HTML

   Neural networks have been successfully applied in modeling partial differential equations, especially in dynamical systems. Commonly used models, such as neural operators, are performing well at deterministic prediction tasks, but lack a quantification of the uncertainty inherent in many complex systems, for example weather forecasting. In this paper, we explore a new approach that combines Fourier neural operators with generative modeling based on strictly proper scoring rules in order to create well-calibrated probabilistic predictions of dynamical systems. We demonstrate improved predictive uncertainty for our approach, especially in settings with very high inherent uncertainty.

   @article{prob_predictions_neural_operator,
     title = {Probabilistic predictions with Fourier neural operators},
     author = {Bülte, Christopher and Philipp, Scholl and Gitta, Kutyniok},
     journal = {NeurIPS 2024 workshop on Bayesian Decision-making and Uncertainty},
     url = {https://openreview.net/forum?id=orKA6gJwlB},
     year = {2024},
   }

2. arXiv

   

   Estimation of Spatio-Temporal Extremes via Generative Neural Networks

   Christopher Bülte, Lisa Leimenstoll, and Melanie Schienle

   arXiv preprint arXiv:2407.08668, 2024

   Abs Bib HTML Code

   Recent methods in modeling spatial extreme events have focused on utilizing parametric max-stable processes and their underlying dependence structure. In this work, we provide a unified approach for analyzing spatial extremes with little available data by estimating the distribution of model parameters or the spatial dependence directly. By employing recent developments in generative neural networks we predict a full sample-based distribution, allowing for direct assessment of uncertainty regarding model parameters or other parameter dependent functionals. We validate our method by fitting several simulated max-stable processes, showing a high accuracy of the approach, regarding parameter estimation, as well as uncertainty quantification. Additional robustness checks highlight the generalization and extrapolation capabilities of the model, while an application to precipitation extremes across Western Germany demonstrates the usability of our approach in real-world scenarios.

   @article{bulteEstimationSpatiotemporalExtremes2024,
     title = {Estimation of Spatio-Temporal Extremes via Generative Neural Networks},
     author = {Bülte, Christopher and Leimenstoll, Lisa and Schienle, Melanie},
     journal = {arXiv preprint arXiv:2407.08668},
     year = {2024},
     eprint = {2407.08668},
     eprinttype = {arXiv},
     eprintclass = {cs, stat},
     url = {http://arxiv.org/abs/2407.08668},
     urldate = {2024-07-13},
     keywords = {Computer Science - Machine Learning,Statistics - Machine Learning},
   }

3. arXiv

   

   Uncertainty Quantification for Data-Driven Weather Models

   Christopher Bülte, Nina Horat, Julian Quinting, and Sebastian Lerch

   arXiv preprint arXiv:2403.13458, 2024

   Abs Bib HTML Code

   Artificial intelligence (AI)-based data-driven weather forecasting models have experienced rapid progress over the last years. Recent studies, with models trained on reanalysis data, achieve impressive results and demonstrate substantial improvements over state-of-the-art physics-based numerical weather prediction models across a range of variables and evaluation metrics. Beyond improved predictions, the main advantages of data-driven weather models are their substantially lower computational costs and the faster generation of forecasts, once a model has been trained. However, most efforts in data-driven weather forecasting have been limited to deterministic, point-valued predictions, making it impossible to quantify forecast uncertainties, which is crucial in research and for optimal decision making in applications. Our overarching aim is to systematically study and compare uncertainty quantification methods to generate probabilistic weather forecasts from a state-of-the-art deterministic data-driven weather model, Pangu-Weather. Specifically, we compare approaches for quantifying forecast uncertainty based on generating ensemble forecasts via perturbations to the initial conditions, with the use of statistical and machine learning methods for post-hoc uncertainty quantification. In a case study on medium-range forecasts of selected weather variables over Europe, the probabilistic forecasts obtained by using the Pangu-Weather model in concert with uncertainty quantification methods show promising results and provide improvements over ensemble forecasts from the physics-based ensemble weather model of the European Centre for Medium-Range Weather Forecasts for lead times of up to 5 days.

   @article{bulteUncertaintyQuantificationDatadriven2024,
     title = {Uncertainty Quantification for Data-Driven Weather Models},
     author = {Bülte, Christopher and Horat, Nina and Quinting, Julian and Lerch, Sebastian},
     journal = {arXiv preprint arXiv:2403.13458},
     year = {2024},
     eprint = {2403.13458},
     eprinttype = {arXiv},
     eprintclass = {physics, stat},
     doi = {10.48550/arXiv.2403.13458},
     url = {http://arxiv.org/abs/2403.13458},
     urldate = {2024-04-04},
     pubstate = {prepublished},
     keywords = {Physics - Atmospheric and Oceanic Physics,Statistics - Applications,Statistics - Machine Learning},
   }

2023

1. Energy & AI

   

   Multivariate Time Series Imputation for Energy Data Using Neural Networks

   Christopher Bülte, Max Kleinebrahm, Hasan Ümitcan Yilmaz, and Juan Gómez-Romero

   Energy and AI, 2023

   Abs Bib HTML Code

   Multivariate time series with missing values are common in a wide range of applications, including energy data. Existing imputation methods often fail to focus on the temporal dynamics and the cross-dimensional correlation simultaneously. In this paper we propose a two-step method based on an attention model to impute missing values in multivariate energy time series. First, the underlying distribution of the missing values in the data is learned. This information is then further used to train an attention based imputation model. By learning the distribution prior to the imputation process, the model can respond flexibly to the specific characteristics of the underlying data. The developed model is applied to European energy data, obtained from the European Network of Transmission System Operators for Electricity. Using different evaluation metrics and benchmarks, the conducted experiments show that the proposed model is preferable to the benchmarks and is able to accurately impute missing values.

   @article{bulteMultivariateTimeSeries2023,
     title = {Multivariate Time Series Imputation for Energy Data Using Neural Networks},
     author = {Bülte, Christopher and Kleinebrahm, Max and Yilmaz, Hasan Ümitcan and Gómez-Romero, Juan},
     year = {2023},
     journal = {Energy and AI},
     volume = {13},
     pages = {100239},
     issn = {2666-5468},
     doi = {10.1016/j.egyai.2023.100239},
     url = {https://www.sciencedirect.com/science/article/pii/S2666546823000113},
     keywords = {Attention model,Energy data,Missing value estimation,Multivariate time series,Neural networks},
   }

2021

1. ICML 2021

   

   Applying transformer to imputation of multivariate energy time series data

   Hasan Ümitcan Yilmaz, Max Kleinebrahm, Christopher Bülte, and Juan Gómez-Romero

   ICML 2021 Workshop on Tackling Climate Change with Machine Learning, 2021

   Abs Bib HTML Slides

   To reduce the greenhouse gas emissions from electricity production, it is necessary to switch to an energy system based on renewable energy sources (RES). However, intermittent electricity generation from RES poses challenges for energy systems. The primary input for data-driven solutions is data on electricity generation from RES, whichusually contain many missing values. This proposal studies the use of attention-based algorithms to impute missing values of electricity production,electricity demand and electricity prices. Since attention mechanisms allow us to take into account dependencies between time series across multiple dimensions efficiently, our approach goes beyond classic statistical methods and incorporates many related variables, such as electricity price, demand and production by other sources. Our preliminary results show that while transformers can come at higher computational costs, they are more precise than classical imputation methods.

   @article{yilmaz2021applying,
     title = {Applying transformer to imputation of multivariate energy time series data},
     author = {Yilmaz, Hasan Ümitcan and Kleinebrahm, Max and Bülte, Christopher and Gómez-Romero, Juan},
     journal = {ICML 2021 Workshop on Tackling Climate Change with Machine Learning},
     url = {https://www.climatechange.ai/papers/icml2021/89},
     year = {2021},
   }