%0 Conference Paper
%A Villamar, Jose
%A Kelbling, Matthias
%A Terhorst, Dennis
%A More, Heather
%A Tetzlaff, Tom
%A Senk, Johanna
%A Thober, Stephan
%T Tracking large-scale simulations through unified metadata handling
%M FZJ-2023-03229
%D 2022
%X Simulation is an essential pillar of knowledge generation in science. The numerical models used to describe, predict, and understand real-world systems are typically complex. Consequently,  applying these models by means of simulation often poses high demands on computational resources, and requires high-performance computing (HPC) or other dedicated hardware architectures. Metadata describing the details of a numerical experiment arise at all stages of the simulation process: the conceptual description of the model, the model implementation, and the tools and machines used to run the simulation. Capturing these metadata and provenance information along the processing chain is a vital requirement for several purposes, e.g. reproducibility, benchmarking and validation, assessment of the reliability of the simulations, and data exploration [1,2]. The ability to search, share, and evaluate metadata and provenance traces from heterogeneous simulations and environments is a major challenge in provenance-driven analysis. The availability of a common metadata framework, which can be adopted by scientists from different scientific domains, would foster the meta-analysis of HPC simulation workflows [3]. Here, we develop a metadata management framework for generic HPC-based simulation research comprising concepts and tools for efficiently generating, organizing, and exploring metadata along a given simulation workflow.  The derived solutions cope with the modularity and flexibility demands of rapidly progressing science and are applicable to diverse research fields. As a proof of concept, we will apply these solutions to use cases from environmental research and computational neuroscience.[1] Guilyardi, E., et. al. (2013) doi: 10.1175/BAMS-D-11-00035.1[2] Manninen, T., et. al. (2018) doi: 10.3389/fninf.2018.00020[3] Ivie, P., & Thain, D. (2018). doi: 10.1145/3186266
%B Helmholtz Metadata Collaboration conference
%C 5 Oct 2022 - 6 Oct 2022, Virtual (Germany)
Y2 5 Oct 2022 - 6 Oct 2022
M2 Virtual, Germany
%F PUB:(DE-HGF)24
%9 Poster
%U https://juser.fz-juelich.de/record/1014307