Publication details

Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations

Authors

SHERRATT Katharine GRUSON Hugo GRAH Rok JOHNSON Helen NIEHUS Rene PRASSE Bastian SANDMANN Frank DEUSCHEL Jannik WOLFFRAM Daniel ABBOTT Sam ULLRICH Alexander GIBSON Graham RAY Evan L REICH Nicholas G SHELDON Daniel WANG Yijin WATTANACHIT Nutcha WANG Lijing TRNKA Jan OBOZINSKI Guillaume SUN Tao THANOU Dorina POTTIER Loic KRYMOVA Ekaterina MEINKE Jan H BARBAROSSA Maria Vittoria LEITHÄUSER Neele MOHRING Jan SCHNEIDER Johanna WŁAZŁO Jaroslaw FUHRMANN Jan LANGE Berit RODIAH Isti BACCAM Prasith GURUNG Heidi STAGE Steven SUCHOSKI Bradley BUDZINSKI Jozef WALRAVEN Robert VILLANUEVA Inmaculada TUCEK Vit SMID Martin ZAJÍČEK Milan ÁLVAREZ Cesar Pérez REINA Borja BOSSE Nikos I MEAKIN Sophie R CASTRO Lauren FAIRCHILD Geoffrey MICHAUD Isaac OSTHUS Dave LORO Pierfrancesco Alaimo Di MARUOTTI Antonello ECLEROVÁ Veronika KRAUS Andrea KRAUS David PŘIBYLOVÁ Lenka DIMITRIS Bertsimas LI Michael Lingzhi SAKSHAM Soni DEHNING Jonas MOHR Sebastian PRIESEMANN Viola REDLARSKI Grzegorz BEJAR Benjamin ARDENGHI Giovanni PAROLINI Nicola ZIARELLI Giovanni BOCK Wolfgang HEYDER Stefan HOTZ Thomas SINGH David E GUZMAN-MERINO Miguel AZNARTE Jose L MORINA David ALONSO Sergio ÁLVAREZ Enric LÓPEZ Daniel PRATS Clara BURGARD Jan Pablo RODLOFF Arne ZIMMERMANN Tom KUHLMANN Alexander ZIBERT Janez PENNONI Fulvia DIVINO Fabio CATALA Marti LOVISON Gianfranco GIUDICI Paolo TARANTINO Barbara BARTOLUCCI Francesco LASINIO Giovanna Jona MINGIONE Marco FARCOMENI Alessio SRIVASTAVA Ajitesh MONTERO-MANSO Pablo ADIGA Aniruddha HURT Benjamin LEWIS Bryan MARATHE Madhav POREBSKI Przemyslaw VENKATRAMANAN Srinivasan BARTCZUK Rafal P DREGER Filip GAMBIN Anna GOGOLEWSKI Krzysztof GRUZIEL-SŁOMKA Magdalena KRUPA Bartosz MOSZYŃSKI Antoni NIEDZIELEWSKI Karol NOWOSIELSKI Jedrzej RADWAN Maciej RAKOWSKI Franciszek SEMENIUK Marcin SZCZUREK Ewa ZIELIŃSKI Jakub KISIELEWSKI Jan PABJAN Barbara KIRSTEN Holger KHEIFETZ Yuri SCHOLZ Markus BIECEK Przemyslaw BODYCH Marcin FILINSKI Maciej IDZIKOWSKI Radoslaw KRUEGER Tyll OZANSKI Tomasz BRACHER Johannes FUNK Sebastian

Year of publication 2023
Type Article in Periodical
Magazine / Source eLife
MU Faculty or unit

Faculty of Science

Citation
Web https://doi.org/10.7554/eLife.81916
Doi http://dx.doi.org/10.7554/eLife.81916
Keywords modelling; forecast; COVID-19; Europe; ensemble; prediction
Description Background: Short-term forecasts of infectious disease contribute to situational awareness and capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise forecasts’ predictive performance by combining independent models into an ensemble. Here we report the performance of ensemble predictions of COVID-19 cases and deaths across Europe from March 2021 to March 2022. Methods: We created the European COVID-19 Forecast Hub, an online open-access platform where modellers upload weekly forecasts for 32 countries with results publicly visualised and evaluated. We created a weekly ensemble forecast from the equally-weighted average across individual models' predictive quantiles. We measured forecast accuracy using a baseline and relative Weighted Interval Score (rWIS). We retrospectively explored ensemble methods, including weighting by past performance. Results: We collected weekly forecasts from 48 models, of which we evaluated 29 models alongside the ensemble model. The ensemble had a consistently strong performance across countries over time, performing better on rWIS than 91% of forecasts for deaths (N=763 predictions from 20 models), and 83% forecasts for cases (N=886 predictions from 23 models). Performance remained stable over a 4-week horizon for death forecasts but declined with longer horizons for cases. Among ensemble methods, the most influential choice came from using a median average instead of the mean, regardless of weighting component models.
Related projects:

You are running an old browser version. We recommend updating your browser to its latest version.

More info