Staged deployment of interactive multi-application HPC workflows

Klijn, Wouter; Diaz, Sandra; Peyser, Alexander; Morrison, Abigail

doi:10.1109/HPCS48598.2019.9188104

Contribution to a conference proceedings

FZJ-2019-04310

Staged deployment of interactive multi-application HPC workflows

Klijn, W. (Corresponding author)FZJ* ; Diaz, S.FZJ* ; Morrison, A.FZJ* ; Peyser, A.FZJ*

2020
IEEE

The 2019 International Conference on High Performance Computing & Simulation, HPCS 2019, Dublin, Ireland, 15 Jul 2019 - 19 Jul 2019 IEEE 305-311 (2020) [10.1109/HPCS48598.2019.9188104]

This record in other databases:

Please use a persistent id in citations: http://hdl.handle.net/2128/25761 doi:10.1109/HPCS48598.2019.9188104

Abstract: Running scientific workflows on a supercomputer can be a daunting task for a scientific domain specialist. Workflow management solutions (WMS) are a standard method for reducing the complexity of application deployment on high performance computing (HPC) infrastructure. We introduce the design for a middleware system that extends and combines the functionality from existing solutions in order to create a high-level, staged user-centric operation/deployment model. This design addresses the requirements of several use cases in the life sciences, with a focus on neuroscience. In this manuscript we focus on two use cases: 1) three coupled neuronal simulators (for three different space/time scales) with in-transit visualization and 2) a closed-loop workflow optimized by machine learning, coupling a robot with a neural network simulation. We provide a detailed overview of the application-integrated monitoring in relationship with the HPC job. We present here a novel usage model for large scale interactive multi-application workflows running on HPC systems which aims at reducing the complexity of deployment and execution, thus enabling new science.

Contributing Institute(s):

Research Program(s):

Appears in the scientific report 2020

Database coverage:
OpenAccess