001009058 001__ 1009058
001009058 005__ 20230929112538.0
001009058 0247_ $$2doi$$a10.7554/eLife.81916
001009058 0247_ $$2datacite_doi$$a10.34734/FZJ-2023-02606
001009058 0247_ $$2pmid$$a37083521
001009058 0247_ $$2WOS$$aWOS:001009734700001
001009058 037__ $$aFZJ-2023-02606
001009058 082__ $$a600
001009058 1001_ $$00000-0003-2049-3423$$aSherratt, Katharine$$b0$$eCorresponding author
001009058 245__ $$aPredictive performance of multi-model ensemble forecasts of COVID-19 across European nations
001009058 260__ $$aCambridge$$beLife Sciences Publications$$c2023
001009058 3367_ $$2DRIVER$$aarticle
001009058 3367_ $$2DataCite$$aOutput Types/Journal article
001009058 3367_ $$0PUB:(DE-HGF)16$$2PUB:(DE-HGF)$$aJournal Article$$bjournal$$mjournal$$s1689667582_4435
001009058 3367_ $$2BibTeX$$aARTICLE
001009058 3367_ $$2ORCID$$aJOURNAL_ARTICLE
001009058 3367_ $$00$$2EndNote$$aJournal Article
001009058 520__ $$aBackground:Short-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here, we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022.Methods:We used open-source tools to develop a public European COVID-19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID-19 cases and deaths reported by a standardised source for 32 countries over the next 1–4 weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equally-weighted average (initially the mean and then from 26th July the median) of all individual models’ predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models’ forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models’ past predictive performance.Results:Over 52 weeks, we collected forecasts from 48 unique models. We evaluated 29 models’ forecast scores in comparison to the ensemble model. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 83% of participating models’ forecasts of incident cases (with a total N=886 predictions from 23 unique models), and 91% of participating models’ forecasts of deaths (N=763 predictions from 20 models). Across a 1–4 week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over 4 weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models.Conclusions:Our results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than 2 weeks.
001009058 536__ $$0G:(DE-HGF)POF4-5111$$a5111 - Domain-Specific Simulation & Data Life Cycle Labs (SDLs) and Research Groups (POF4-511)$$cPOF4-511$$fPOF IV$$x0
001009058 588__ $$aDataset connected to CrossRef, Journals: juser.fz-juelich.de
001009058 7001_ $$0P:(DE-HGF)0$$aGruson, Hugo$$b1
001009058 7001_ $$0P:(DE-HGF)0$$aGrah, Rok$$b2
001009058 7001_ $$0P:(DE-HGF)0$$aJohnson, Helen$$b3
001009058 7001_ $$0P:(DE-HGF)0$$aNiehus, Rene$$b4
001009058 7001_ $$0P:(DE-HGF)0$$aPrasse, Bastian$$b5
001009058 7001_ $$0P:(DE-HGF)0$$aSandmann, Frank$$b6
001009058 7001_ $$0P:(DE-HGF)0$$aDeuschel, Jannik$$b7
001009058 7001_ $$00000-0003-0318-3669$$aWolffram, Daniel$$b8
001009058 7001_ $$0P:(DE-HGF)0$$aAbbott, Sam$$b9
001009058 7001_ $$0P:(DE-HGF)0$$aUllrich, Alexander$$b10
001009058 7001_ $$0P:(DE-HGF)0$$aGibson, Graham$$b11
001009058 7001_ $$0P:(DE-HGF)0$$aRay, Evan L$$b12
001009058 7001_ $$0P:(DE-HGF)0$$aReich, Nicholas G$$b13
001009058 7001_ $$0P:(DE-HGF)0$$aSheldon, Daniel$$b14
001009058 7001_ $$00000-0003-4438-6366$$aWang, Yijin$$b15
001009058 7001_ $$0P:(DE-HGF)0$$aWattanachit, Nutcha$$b16
001009058 7001_ $$0P:(DE-HGF)0$$aWang, Lijing$$b17
001009058 7001_ $$00000-0002-1786-7562$$aTrnka, Jan$$b18
001009058 7001_ $$0P:(DE-HGF)0$$aObozinski, Guillaume$$b19
001009058 7001_ $$00000-0001-6357-6726$$aSun, Tao$$b20
001009058 7001_ $$0P:(DE-HGF)0$$aThanou, Dorina$$b21
001009058 7001_ $$0P:(DE-HGF)0$$aPottier, Loic$$b22
001009058 7001_ $$0P:(DE-HGF)0$$aKrymova, Ekaterina$$b23
001009058 7001_ $$0P:(DE-Juel1)132189$$aMeinke, Jan H$$b24
001009058 7001_ $$0P:(DE-HGF)0$$aBarbarossa, Maria Vittoria$$b25
001009058 7001_ $$0P:(DE-HGF)0$$aLeithauser, Neele$$b26
001009058 7001_ $$0P:(DE-HGF)0$$aMohring, Jan$$b27
001009058 7001_ $$00000-0002-9330-2838$$aSchneider, Johanna$$b28
001009058 7001_ $$0P:(DE-HGF)0$$aWlazlo, Jaroslaw$$b29
001009058 7001_ $$0P:(DE-Juel1)184603$$aFuhrmann, Jan$$b30
001009058 7001_ $$0P:(DE-HGF)0$$aLange, Berit$$b31
001009058 7001_ $$0P:(DE-HGF)0$$aRodiah, Isti$$b32
001009058 7001_ $$0P:(DE-HGF)0$$aBaccam, Prasith$$b33
001009058 7001_ $$0P:(DE-HGF)0$$aGurung, Heidi$$b34
001009058 7001_ $$0P:(DE-HGF)0$$aStage, Steven$$b35
001009058 7001_ $$0P:(DE-HGF)0$$aSuchoski, Bradley$$b36
001009058 7001_ $$0P:(DE-HGF)0$$aBudzinski, Jozef$$b37
001009058 7001_ $$0P:(DE-HGF)0$$aWalraven, Robert$$b38
001009058 7001_ $$00000-0003-4940-085X$$aVillanueva, Inmaculada$$b39
001009058 7001_ $$0P:(DE-HGF)0$$aTucek, Vit$$b40
001009058 7001_ $$0P:(DE-HGF)0$$aSmid, Martin$$b41
001009058 7001_ $$00000-0002-3226-7266$$aZajicek, Milan$$b42
001009058 7001_ $$0P:(DE-HGF)0$$aPerez Alvarez, Cesar$$b43
001009058 7001_ $$0P:(DE-HGF)0$$aReina, Borja$$b44
001009058 7001_ $$0P:(DE-HGF)0$$aBosse, Nikos I$$b45
001009058 7001_ $$0P:(DE-HGF)0$$aMeakin, Sophie R$$b46
001009058 7001_ $$0P:(DE-HGF)0$$aCastro, Lauren$$b47
001009058 7001_ $$0P:(DE-HGF)0$$aFairchild, Geoffrey$$b48
001009058 7001_ $$0P:(DE-HGF)0$$aMichaud, Isaac$$b49
001009058 7001_ $$0P:(DE-HGF)0$$aOsthus, Dave$$b50
001009058 7001_ $$0P:(DE-HGF)0$$aAlaimo Di Loro, Pierfrancesco$$b51
001009058 7001_ $$00000-0001-8377-9950$$aMaruotti, Antonello$$b52
001009058 7001_ $$00000-0001-8476-7740$$aEclerova, Veronika$$b53
001009058 7001_ $$0P:(DE-HGF)0$$aKraus, Andrea$$b54
001009058 7001_ $$0P:(DE-HGF)0$$aKraus, David$$b55
001009058 7001_ $$0P:(DE-HGF)0$$aPribylova, Lenka$$b56
001009058 7001_ $$0P:(DE-HGF)0$$aDimitris, Bertsimas$$b57
001009058 7001_ $$0P:(DE-HGF)0$$aLi, Michael Lingzhi$$b58
001009058 7001_ $$0P:(DE-HGF)0$$aSaksham, Soni$$b59
001009058 7001_ $$0P:(DE-HGF)0$$aDehning, Jonas$$b60
001009058 7001_ $$0P:(DE-HGF)0$$aMohr, Sebastian$$b61
001009058 7001_ $$00000-0001-8905-5873$$aPriesemann, Viola$$b62
001009058 7001_ $$0P:(DE-HGF)0$$aRedlarski, Grzegorz$$b63
001009058 7001_ $$0P:(DE-HGF)0$$aBejar, Benjamin$$b64
001009058 7001_ $$0P:(DE-HGF)0$$aArdenghi, Giovanni$$b65
001009058 7001_ $$0P:(DE-HGF)0$$aParolini, Nicola$$b66
001009058 7001_ $$0P:(DE-HGF)0$$aZiarelli, Giovanni$$b67
001009058 7001_ $$0P:(DE-HGF)0$$aBock, Wolfgang$$b68
001009058 7001_ $$0P:(DE-HGF)0$$aHeyder, Stefan$$b69
001009058 7001_ $$0P:(DE-HGF)0$$aHotz, Thomas$$b70
001009058 7001_ $$0P:(DE-HGF)0$$aSingh, David E$$b71
001009058 7001_ $$0P:(DE-HGF)0$$aGuzman-Merino, Miguel$$b72
001009058 7001_ $$0P:(DE-HGF)0$$aAznarte, Jose L$$b73
001009058 7001_ $$0P:(DE-HGF)0$$aMorina, David$$b74
001009058 7001_ $$00000-0002-3989-8757$$aAlonso, Sergio$$b75
001009058 7001_ $$0P:(DE-HGF)0$$aAlvarez, Enric$$b76
001009058 7001_ $$0P:(DE-HGF)0$$aLopez, Daniel$$b77
001009058 7001_ $$00000-0002-1398-7559$$aPrats, Clara$$b78
001009058 7001_ $$00000-0002-5771-6179$$aBurgard, Jan Pablo$$b79
001009058 7001_ $$0P:(DE-HGF)0$$aRodloff, Arne$$b80
001009058 7001_ $$0P:(DE-HGF)0$$aZimmermann, Tom$$b81
001009058 7001_ $$0P:(DE-HGF)0$$aKuhlmann, Alexander$$b82
001009058 7001_ $$0P:(DE-HGF)0$$aZibert, Janez$$b83
001009058 7001_ $$0P:(DE-HGF)0$$aPennoni, Fulvia$$b84
001009058 7001_ $$0P:(DE-HGF)0$$aDivino, Fabio$$b85
001009058 7001_ $$0P:(DE-HGF)0$$aCatala, Marti$$b86
001009058 7001_ $$0P:(DE-HGF)0$$aLovison, Gianfranco$$b87
001009058 7001_ $$0P:(DE-HGF)0$$aGiudici, Paolo$$b88
001009058 7001_ $$0P:(DE-HGF)0$$aTarantino, Barbara$$b89
001009058 7001_ $$0P:(DE-HGF)0$$aBartolucci, Francesco$$b90
001009058 7001_ $$0P:(DE-HGF)0$$aJona Lasinio, Giovanna$$b91
001009058 7001_ $$0P:(DE-HGF)0$$aMingione, Marco$$b92
001009058 7001_ $$00000-0002-7104-5826$$aFarcomeni, Alessio$$b93
001009058 7001_ $$0P:(DE-HGF)0$$aSrivastava, Ajitesh$$b94
001009058 7001_ $$0P:(DE-HGF)0$$aMontero-Manso, Pablo$$b95
001009058 7001_ $$0P:(DE-HGF)0$$aAdiga, Aniruddha$$b96
001009058 7001_ $$0P:(DE-HGF)0$$aHurt, Benjamin$$b97
001009058 7001_ $$00000-0003-0793-6082$$aLewis, Bryan$$b98
001009058 7001_ $$0P:(DE-HGF)0$$aMarathe, Madhav$$b99
001009058 7001_ $$00000-0001-8012-5791$$aPorebski, Przemyslaw$$b100
001009058 7001_ $$0P:(DE-HGF)0$$aVenkatramanan, Srinivasan$$b101
001009058 7001_ $$00000-0002-0433-7327$$aBartczuk, Rafal P$$b102
001009058 7001_ $$0P:(DE-HGF)0$$aDreger, Filip$$b103
001009058 7001_ $$0P:(DE-HGF)0$$aGambin, Anna$$b104
001009058 7001_ $$00000-0001-5523-5198$$aGogolewski, Krzysztof$$b105
001009058 7001_ $$0P:(DE-HGF)0$$aGruziel-Slomka, Magdalena$$b106
001009058 7001_ $$0P:(DE-HGF)0$$aKrupa, Bartosz$$b107
001009058 7001_ $$0P:(DE-HGF)0$$aMoszyński, Antoni$$b108
001009058 7001_ $$0P:(DE-HGF)0$$aNiedzielewski, Karol$$b109
001009058 7001_ $$0P:(DE-HGF)0$$aNowosielski, Jedrzej$$b110
001009058 7001_ $$0P:(DE-HGF)0$$aRadwan, Maciej$$b111
001009058 7001_ $$0P:(DE-HGF)0$$aRakowski, Franciszek$$b112
001009058 7001_ $$0P:(DE-HGF)0$$aSemeniuk, Marcin$$b113
001009058 7001_ $$0P:(DE-HGF)0$$aSzczurek, Ewa$$b114
001009058 7001_ $$00000-0001-8935-8137$$aZielinski, Jakub$$b115
001009058 7001_ $$0P:(DE-HGF)0$$aKisielewski, Jan$$b116
001009058 7001_ $$0P:(DE-HGF)0$$aPabjan, Barbara$$b117
001009058 7001_ $$0P:(DE-HGF)0$$aHolger, Kirsten$$b118
001009058 7001_ $$0P:(DE-HGF)0$$aKheifetz, Yuri$$b119
001009058 7001_ $$0P:(DE-HGF)0$$aScholz, Markus$$b120
001009058 7001_ $$0P:(DE-HGF)0$$aPrzemyslaw, Biecek$$b121
001009058 7001_ $$0P:(DE-HGF)0$$aBodych, Marcin$$b122
001009058 7001_ $$0P:(DE-HGF)0$$aFilinski, Maciej$$b123
001009058 7001_ $$0P:(DE-HGF)0$$aIdzikowski, Radoslaw$$b124
001009058 7001_ $$0P:(DE-HGF)0$$aKrueger, Tyll$$b125
001009058 7001_ $$0P:(DE-HGF)0$$aOzanski, Tomasz$$b126
001009058 7001_ $$0P:(DE-HGF)0$$aBracher, Johannes$$b127
001009058 7001_ $$00000-0002-2842-3406$$aFunk, Sebastian$$b128
001009058 773__ $$0PERI:(DE-600)2687154-3$$a10.7554/eLife.81916$$gVol. 12, p. e81916$$pe81916$$teLife$$v12$$x2050-084X$$y2023
001009058 8564_ $$uhttps://juser.fz-juelich.de/record/1009058/files/Sherratt%20et%20al.%20-%202023%20-%20Predictive%20performance%20of%20multi-model%20ensemble%20for.pdf$$yOpenAccess
001009058 909CO $$ooai:juser.fz-juelich.de:1009058$$pdnbdelivery$$pdriver$$pVDB$$popen_access$$popenaire
001009058 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)132189$$aForschungszentrum Jülich$$b24$$kFZJ
001009058 9131_ $$0G:(DE-HGF)POF4-511$$1G:(DE-HGF)POF4-510$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5111$$aDE-HGF$$bKey Technologies$$lEngineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action$$vEnabling Computational- & Data-Intensive Science and Engineering$$x0
001009058 9141_ $$y2023
001009058 915__ $$0StatID:(DE-HGF)0160$$2StatID$$aDBCoverage$$bEssential Science Indicators$$d2022-11-23
001009058 915__ $$0StatID:(DE-HGF)1190$$2StatID$$aDBCoverage$$bBiological Abstracts$$d2022-11-23
001009058 915__ $$0StatID:(DE-HGF)0501$$2StatID$$aDBCoverage$$bDOAJ Seal$$d2022-09-23T12:20:44Z
001009058 915__ $$0StatID:(DE-HGF)0500$$2StatID$$aDBCoverage$$bDOAJ$$d2022-09-23T12:20:44Z
001009058 915__ $$0StatID:(DE-HGF)0113$$2StatID$$aWoS$$bScience Citation Index Expanded$$d2022-11-23
001009058 915__ $$0StatID:(DE-HGF)0700$$2StatID$$aFees$$d2022-11-23
001009058 915__ $$0StatID:(DE-HGF)0510$$2StatID$$aOpenAccess
001009058 915__ $$0StatID:(DE-HGF)0561$$2StatID$$aArticle Processing Charges$$d2022-11-23
001009058 915__ $$0StatID:(DE-HGF)0100$$2StatID$$aJCR$$bELIFE : 2022$$d2023-08-22
001009058 915__ $$0StatID:(DE-HGF)0200$$2StatID$$aDBCoverage$$bSCOPUS$$d2023-08-22
001009058 915__ $$0StatID:(DE-HGF)0300$$2StatID$$aDBCoverage$$bMedline$$d2023-08-22
001009058 915__ $$0StatID:(DE-HGF)0320$$2StatID$$aDBCoverage$$bPubMed Central$$d2023-08-22
001009058 915__ $$0StatID:(DE-HGF)0030$$2StatID$$aPeer Review$$bDOAJ : Anonymous peer review$$d2022-09-23T12:20:44Z
001009058 915__ $$0StatID:(DE-HGF)0600$$2StatID$$aDBCoverage$$bEbsco Academic Search$$d2023-08-22
001009058 915__ $$0StatID:(DE-HGF)0030$$2StatID$$aPeer Review$$bASC$$d2023-08-22
001009058 915__ $$0StatID:(DE-HGF)0199$$2StatID$$aDBCoverage$$bClarivate Analytics Master Journal List$$d2023-08-22
001009058 915__ $$0StatID:(DE-HGF)1050$$2StatID$$aDBCoverage$$bBIOSIS Previews$$d2023-08-22
001009058 915__ $$0StatID:(DE-HGF)0150$$2StatID$$aDBCoverage$$bWeb of Science Core Collection$$d2023-08-22
001009058 915__ $$0StatID:(DE-HGF)1040$$2StatID$$aDBCoverage$$bZoological Record$$d2023-08-22
001009058 915__ $$0StatID:(DE-HGF)9905$$2StatID$$aIF >= 5$$bELIFE : 2022$$d2023-08-22
001009058 9201_ $$0I:(DE-Juel1)JSC-20090406$$kJSC$$lJülich Supercomputing Center$$x0
001009058 980__ $$ajournal
001009058 980__ $$aVDB
001009058 980__ $$aUNRESTRICTED
001009058 980__ $$aI:(DE-Juel1)JSC-20090406
001009058 9801_ $$aFullTexts