Journal Article

FZJ-2022-01707

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png HPC-oriented Canonical Workflows for Machine Learning Applications in Climate and Weather Prediction

Mozaffari, A. (Corresponding author)FZJ* ; Langguth, M.FZJ* ; Gong, B.FZJ* ; Ahring, J.FZJ* ; Campos, A. R. ; Nieters, P. ; Escobar, O. J. C. ; Wittenbrink, M. ; Baumann, P. ; Schultz, M. G.FZJ*

2022
MIT Press Cambridge, MA

This record in other databases:    

Please use a persistent id in citations: http://hdl.handle.net/2128/31434  doi:10.1162/dint_a_00131

Abstract: Machine learning (ML) applications in weather and climate are gaining momentum as big data and the immense increase in High-performance computing (HPC) power are paving the way. Ensuring FAIR data and reproducible ML practices are significant challenges for Earth system researchers. Even though the FAIR principle is well known to many scientists, research communities are slow to adopt them. Canonical Workflow Framework for Research (CWFR) provides a platform to ensure the FAIRness and reproducibility of these practices without overwhelming researchers. This conceptual paper envisions a holistic CWFR approach towards ML applications in weather and climate, focusing on HPC and big data. Specifically, we discuss Fair Digital Object (FDO) and Research Object (RO) in the DeepRain project to achieve granular reproducibility. DeepRain is a project that aims to improve precipitation forecast in Germany by using ML. Our concept envisages the raster datacube to provide data harmonization and fast and scalable data access. We suggest the Juypter notebook as a single reproducible experiment. In addition, we envision JuypterHub as a scalable and distributed central platform that connects all these elements and the HPC resources to the researchers via an easy-to-use graphical interface.

---

Contributing Institute(s):

1. Jülich Supercomputing Center (JSC)

Research Program(s):

1. 5111 - Domain-Specific Simulation & Data Life Cycle Labs (SDLs) and Research Groups (POF4-511) (POF4-511)
2. Verbundprojekt DeepRain: Effiziente Lokale Niederschlagsvorhersage durch Maschinelles Lernen (01IS18047A) (01IS18047A)
3. Earth System Data Exploration (ESDE) (ESDE)

Appears in the scientific report 2022

---

Database coverage:
; ; ; ; ; Clarivate Analytics Master Journal List ; DOAJ Seal ; Emerging Sources Citation Index ; SCOPUS ; Web of Science Core Collection

---