guest ::
| Home > Publications database > Advanced climate model evaluation with ESMValTool v2.11.0 using parallel, out-of-core, and distributed computing > print |
| Home > Publications database > Advanced climate model evaluation with ESMValTool v2.11.0 using parallel, out-of-core, and distributed computing > print |
| 001 | 1050041 | ||
| 005 | 20251219202235.0 | ||
| 024 | 7 | _ | |a 10.5194/gmd-18-4009-2025 |2 doi |
| 024 | 7 | _ | |a 1991-959X |2 ISSN |
| 024 | 7 | _ | |a 1991-9603 |2 ISSN |
| 024 | 7 | _ | |a 10.34734/FZJ-2025-05755 |2 datacite_doi |
| 037 | _ | _ | |a FZJ-2025-05755 |
| 082 | _ | _ | |a 550 |
| 100 | 1 | _ | |a Schlund, Manuel |0 0000-0001-5251-0158 |b 0 |e Corresponding author |
| 245 | _ | _ | |a Advanced climate model evaluation with ESMValTool v2.11.0 using parallel, out-of-core, and distributed computing |
| 260 | _ | _ | |a Katlenburg-Lindau |c 2025 |b Copernicus |
| 336 | 7 | _ | |a article |2 DRIVER |
| 336 | 7 | _ | |a Output Types/Journal article |2 DataCite |
| 336 | 7 | _ | |a Journal Article |b journal |m journal |0 PUB:(DE-HGF)16 |s 1766155096_15646 |2 PUB:(DE-HGF) |
| 336 | 7 | _ | |a ARTICLE |2 BibTeX |
| 336 | 7 | _ | |a JOURNAL_ARTICLE |2 ORCID |
| 336 | 7 | _ | |a Journal Article |0 0 |2 EndNote |
| 520 | _ | _ | |a Earth system models (ESMs) allow numerical simulations of the Earth's climate system. Driven by the need to better understand climate change and its impacts, these models have become increasingly sophisticated over time, generating vast amounts of data. To effectively evaluate the complex state-of-the-art ESMs and ensure their reliability, new tools for comprehensive analysis are essential. The open-source community-driven Earth System Model Evaluation Tool (ESMValTool) addresses this critical need by providing a software package for scientists to assess the performance of ESMs using common diagnostics and metrics. In this paper, we describe recent significant improvements of ESMValTool's computational efficiency, which allow a more effective evaluation of these complex ESMs and also high-resolution models. These optimizations include parallel computing (execute multiple computation tasks simultaneously), out-of-core computing (process data larger than available memory), and distributed computing (spread computation tasks across multiple interconnected nodes or machines). When comparing the latest ESMValTool version with a previous not yet optimized version, we find significant performance improvements for many relevant applications running on a single node of a high-performance computing (HPC) system, ranging from 2.3 times faster runs in a multi-model setup up to 23 times faster runs for processing a single high-resolution model. By utilizing distributed computing on two nodes of an HPC system, these speedup factors can be further improved to 3.0 and 44, respectively. Moreover, evaluation runs with the latest version of ESMValTool also require significantly less computational resources than before, which in turn reduces power consumption and thus the overall carbon footprint of ESMValTool runs. For example, the previously mentioned use cases use 2.3 (multi-model evaluation) and 23 (high-resolution model evaluation) times less resources compared to the reference version on one HPC node. Finally, analyses which could previously only be performed on machines with large amounts of memory can now be conducted on much smaller hardware through the use of out-of-core computation. For instance, the high-resolution single-model evaluation use case can now be run with 8 GB of available memory despite an input data size of 35 GB, which was not possible with earlier versions of ESMValTool. This enables running much more complex evaluation tasks on a personal laptop than before. |
| 536 | _ | _ | |a 5111 - Domain-Specific Simulation & Data Life Cycle Labs (SDLs) and Research Groups (POF4-511) |0 G:(DE-HGF)POF4-5111 |c POF4-511 |f POF IV |x 0 |
| 536 | _ | _ | |a 2A5 - Exascale Earth System Modeling (CARF - CCA) (POF4-2A5) |0 G:(DE-HGF)POF4-2A5 |c POF4-2A5 |f POF IV |x 1 |
| 536 | _ | _ | |a USMILE - Understanding and Modelling the Earth System with Machine Learning (855187) |0 G:(EU-Grant)855187 |c 855187 |f ERC-2019-SyG |x 2 |
| 536 | _ | _ | |a ESM2025 - Earth system models for the future (101003536) |0 G:(EU-Grant)101003536 |c 101003536 |f H2020-LC-CLA-2020-2 |x 3 |
| 536 | _ | _ | |a IS-ENES3 - Infrastructure for the European Network for Earth System modelling - Phase 3 (824084) |0 G:(EU-Grant)824084 |c 824084 |f H2020-INFRAIA-2018-1 |x 4 |
| 536 | _ | _ | |a EUCP - European Climate Prediction system (776613) |0 G:(EU-Grant)776613 |c 776613 |f H2020-SC5-2017-OneStageB |x 5 |
| 536 | _ | _ | |a ESiWACE3 - Center of excellence for weather and climate phase 3 (101093054) |0 G:(EU-Grant)101093054 |c 101093054 |f HORIZON-EUROHPC-JU-2021-COE-01 |x 6 |
| 588 | _ | _ | |a Dataset connected to CrossRef, Journals: juser.fz-juelich.de |
| 700 | 1 | _ | |a Andela, Bouwe |0 0000-0001-9005-8940 |b 1 |
| 700 | 1 | _ | |a Benke, Jörg |0 P:(DE-Juel1)185943 |b 2 |u fzj |
| 700 | 1 | _ | |a Comer, Ruth |0 P:(DE-HGF)0 |b 3 |
| 700 | 1 | _ | |a Hassler, Birgit |0 0000-0003-2724-709X |b 4 |
| 700 | 1 | _ | |a Hogan, Emma |0 P:(DE-HGF)0 |b 5 |
| 700 | 1 | _ | |a Kalverla, Peter |0 0000-0002-5025-7862 |b 6 |
| 700 | 1 | _ | |a Lauer, Axel |0 0000-0002-9270-1044 |b 7 |
| 700 | 1 | _ | |a Little, Bill |0 P:(DE-HGF)0 |b 8 |
| 700 | 1 | _ | |a Loosveldt Tomas, Saskia |0 P:(DE-HGF)0 |b 9 |
| 700 | 1 | _ | |a Nattino, Francesco |0 0000-0003-3286-0139 |b 10 |
| 700 | 1 | _ | |a Peglar, Patrick |0 P:(DE-HGF)0 |b 11 |
| 700 | 1 | _ | |a Predoi, Valeriu |0 P:(DE-HGF)0 |b 12 |
| 700 | 1 | _ | |a Smeets, Stef |0 P:(DE-HGF)0 |b 13 |
| 700 | 1 | _ | |a Worsley, Stephen |0 P:(DE-HGF)0 |b 14 |
| 700 | 1 | _ | |a Yeo, Martin |0 0009-0000-6857-560X |b 15 |
| 700 | 1 | _ | |a Zimmermann, Klaus |0 P:(DE-HGF)0 |b 16 |
| 773 | _ | _ | |a 10.5194/gmd-18-4009-2025 |g Vol. 18, no. 13, p. 4009 - 4021 |0 PERI:(DE-600)2456725-5 |n 13 |p 4009 - 4021 |t Geoscientific model development |v 18 |y 2025 |x 1991-959X |
| 856 | 4 | _ | |u https://juser.fz-juelich.de/record/1050041/files/gmd-18-4009-2025.pdf |y OpenAccess |
| 909 | C | O | |o oai:juser.fz-juelich.de:1050041 |p openaire |p open_access |p driver |p VDB |p ec_fundedresources |p dnbdelivery |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 2 |6 P:(DE-Juel1)185943 |
| 913 | 1 | _ | |a DE-HGF |b Key Technologies |l Engineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action |1 G:(DE-HGF)POF4-510 |0 G:(DE-HGF)POF4-511 |3 G:(DE-HGF)POF4 |2 G:(DE-HGF)POF4-500 |4 G:(DE-HGF)POF |v Enabling Computational- & Data-Intensive Science and Engineering |9 G:(DE-HGF)POF4-5111 |x 0 |
| 913 | 1 | _ | |a DE-HGF |b Forschungsbereich Erde und Umwelt |l COOPERATION ACROSS RESEARCH FIELDS (CARFs) |1 G:(DE-HGF)POF4-2A0 |0 G:(DE-HGF)POF4-2A5 |3 G:(DE-HGF)POF4 |2 G:(DE-HGF)POF4-200 |4 G:(DE-HGF)POF |v Exascale Earth System Modeling (CARF - CCA) |x 1 |
| 914 | 1 | _ | |y 2025 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0200 |2 StatID |b SCOPUS |d 2024-12-21 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0160 |2 StatID |b Essential Science Indicators |d 2024-12-21 |
| 915 | _ | _ | |a Creative Commons Attribution CC BY 4.0 |0 LIC:(DE-HGF)CCBY4 |2 HGFVOC |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0600 |2 StatID |b Ebsco Academic Search |d 2024-12-21 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)1150 |2 StatID |b Current Contents - Physical, Chemical and Earth Sciences |d 2024-12-21 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0501 |2 StatID |b DOAJ Seal |d 2022-12-20T09:29:04Z |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0500 |2 StatID |b DOAJ |d 2022-12-20T09:29:04Z |
| 915 | _ | _ | |a WoS |0 StatID:(DE-HGF)0113 |2 StatID |b Science Citation Index Expanded |d 2024-12-21 |
| 915 | _ | _ | |a Fees |0 StatID:(DE-HGF)0700 |2 StatID |d 2024-12-21 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0150 |2 StatID |b Web of Science Core Collection |d 2024-12-21 |
| 915 | _ | _ | |a OpenAccess |0 StatID:(DE-HGF)0510 |2 StatID |
| 915 | _ | _ | |a Peer Review |0 StatID:(DE-HGF)0030 |2 StatID |b ASC |d 2024-12-21 |
| 915 | _ | _ | |a Article Processing Charges |0 StatID:(DE-HGF)0561 |2 StatID |d 2024-12-21 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0300 |2 StatID |b Medline |d 2024-12-21 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0199 |2 StatID |b Clarivate Analytics Master Journal List |d 2024-12-21 |
| 920 | _ | _ | |l yes |
| 920 | 1 | _ | |0 I:(DE-Juel1)JSC-20090406 |k JSC |l Jülich Supercomputing Center |x 0 |
| 980 | _ | _ | |a journal |
| 980 | _ | _ | |a VDB |
| 980 | _ | _ | |a UNRESTRICTED |
| 980 | _ | _ | |a I:(DE-Juel1)JSC-20090406 |
| 980 | 1 | _ | |a FullTexts |
| Library | Collection | CLSMajor | CLSMinor | Language | Author |
|---|