Journal Article

FZJ-2018-05072

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png A run control framework to streamline profiling, porting, and tuning simulation runs and provenance tracking of geoscientific applications

Sharples, W. (Corresponding author)FZJ* ; Zhukov, I.FZJ* ; Geimer, M.FZJ* ; Görgen, K.FZJ* ; Lührs, S.FZJ* ; Breuer, T.FZJ* ; Naz, B.FZJ* ; Kulkarni, K.FZJ* ; Brdar, S.FZJ* ; Kollet, S.FZJ*

2018
Copernicus Katlenburg-Lindau

This record in other databases:    

Please use a persistent id in citations: http://hdl.handle.net/2128/19655  doi:10.5194/gmd-11-2875-2018

Abstract: Geoscientific modeling is constantly evolving, with next-generation geoscientific models and applications placing large demands on high-performance computing (HPC) resources. These demands are being met by new developments in HPC architectures, software libraries, and infrastructures. In addition to the challenge of new massively parallel HPC systems, reproducibility of simulation and analysis results is of great concern. This is due to the fact that next-generation geoscientific models are based on complex model implementations and profiling, modeling, and data processing workflows. Thus, in order to reduce both the duration and the cost of code migration, aid in the development of new models or model components, while ensuring reproducibility and sustainability over the complete data life cycle, an automated approach to profiling, porting, and provenance tracking is necessary. We propose a run control framework (RCF) integrated with a workflow engine as a best practice approach to automate profiling, porting, provenance tracking, and simulation runs. Our RCF encompasses all stages of the modeling chain: (1) preprocess input, (2) compilation of code (including code instrumentation with performance analysis tools), (3) simulation run, and (4) postprocessing and analysis, to address these issues. Within this RCF, the workflow engine is used to create and manage benchmark or simulation parameter combinations and performs the documentation and data organization for reproducibility. In this study, we outline this approach and highlight the subsequent developments scheduled for implementation born out of the extensive profiling of ParFlow. We show that in using our run control framework, testing, benchmarking, profiling, and running models is less time consuming and more robust than running geoscientific applications in an ad hoc fashion, resulting in more efficient use of HPC resources, more strategic code development, and enhanced data integrity and reproducibility.

---

Contributing Institute(s):

1. Agrosphäre (IBG-3)
2. Jülich Supercomputing Center (JSC)
3. JARA - HPC (JARA-HPC)

Research Program(s):

1. 255 - Terrestrial Systems: From Observation to Prediction (POF3-255) (POF3-255)
2. 511 - Computational Science and Mathematical Methods (POF3-511) (POF3-511)
3. EoCoE - Energy oriented Centre of Excellence for computer applications (676629) (676629)
4. POP - Performance Optimisation and Productivity (676553) (676553)
5. Scalable Performance Analysis of Large-Scale Parallel Applications (jzam11_20091101) (jzam11_20091101)
6. Water4Enery (jibg31_20160501) (jibg31_20160501)
7. ATMLPP - ATML Parallel Performance (ATMLPP) (ATMLPP)
8. ATMLAO - ATML Application Optimization and User Service Tools (ATMLAO) (ATMLAO)

Appears in the scientific report 2018

---

Database coverage:
; ; ; Current Contents - Physical, Chemical and Earth Sciences ; DOAJ Seal ; Ebsco Academic Search ; IF < 5 ; JCR ; SCOPUS ; Science Citation Index Expanded ; Thomson Reuters Master Journal List ; Web of Science Core Collection

---