000907436 001__ 907436
000907436 005__ 20230123110616.0
000907436 0247_ $$2doi$$a10.1016/j.future.2022.04.014
000907436 0247_ $$2ISSN$$a0167-739X
000907436 0247_ $$2ISSN$$a1872-7115
000907436 0247_ $$2Handle$$a2128/31145
000907436 0247_ $$2altmetric$$aaltmetric:127394423
000907436 0247_ $$2WOS$$aWOS:000808123100004
000907436 037__ $$aFZJ-2022-02034
000907436 041__ $$aEnglish
000907436 082__ $$a004
000907436 1001_ $$00000-0003-4725-5097$$aEjarque, Jorge$$b0$$eCorresponding author
000907436 245__ $$aEnabling dynamic and intelligent workflows for HPC, data analytics, and AI convergence
000907436 260__ $$aAmsterdam [u.a.]$$bElsevier Science$$c2022
000907436 3367_ $$2DRIVER$$aarticle
000907436 3367_ $$2DataCite$$aOutput Types/Journal article
000907436 3367_ $$0PUB:(DE-HGF)16$$2PUB:(DE-HGF)$$aJournal Article$$bjournal$$mjournal$$s1652180156_28215
000907436 3367_ $$2BibTeX$$aARTICLE
000907436 3367_ $$2ORCID$$aJOURNAL_ARTICLE
000907436 3367_ $$00$$2EndNote$$aJournal Article
000907436 520__ $$aThe evolution of High-Performance Computing (HPC) platforms enables the design and execution of progressively larger and more complex workflow applications in these systems. The complexity comes not only from the number of elements that compose the workflows but also from the type of computations they perform. While traditional HPC workflows target simulations and modelling of physical phenomena, current needs require in addition data analytics (DA) and artificial intelligence (AI) tasks. However, the development of these workflows is hampered by the lack of proper programming models and environments that support the integration of HPC, DA, and AI, as well as the lack of tools to easily deploy and execute the workflows in HPC systems. To progress in this direction, this paper presents use cases where complex workflows are required and investigates the main issues to be addressed for the HPC/DA/AI convergence. Based on this study, the paper identifies the challenges of a new workflow platform to manage complex workflows. Finally, it proposes a development approach for such a workflow platform addressing these challenges in two directions: first, by defining a software stack that provides the functionalities to manage these complex workflows; and second, by proposing the HPC Workflow as a Service (HPCWaaS) paradigm, which leverages the software stack to facilitate the reusability of complex workflows in federated HPC infrastructures. Proposals presented in this work are subject to study and development as part of the EuroHPC eFlows4HPC project.
000907436 536__ $$0G:(DE-HGF)POF4-5112$$a5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511)$$cPOF4-511$$fPOF IV$$x0
000907436 588__ $$aDataset connected to CrossRef, Journals: juser.fz-juelich.de
000907436 7001_ $$00000-0003-2941-5499$$aBadia, Rosa M.$$b1
000907436 7001_ $$0P:(DE-HGF)0$$aAlbertin, Loïc$$b2
000907436 7001_ $$00000-0001-5902-6983$$aAloisio, Giovanni$$b3
000907436 7001_ $$0P:(DE-HGF)0$$aBaglione, Enrico$$b4
000907436 7001_ $$00000-0003-2357-7796$$aBecerra, Yolanda$$b5
000907436 7001_ $$00000-0002-3719-5345$$aBoschert, Stefan$$b6
000907436 7001_ $$0P:(DE-HGF)0$$aBerlin, Julian R.$$b7
000907436 7001_ $$00000-0002-0372-2530$$aD’Anca, Alessandro$$b8
000907436 7001_ $$00000-0002-9206-2385$$aElia, Donatello$$b9
000907436 7001_ $$0P:(DE-HGF)0$$aExertier, François$$b10
000907436 7001_ $$0P:(DE-HGF)0$$aFiore, Sandro$$b11
000907436 7001_ $$0P:(DE-HGF)0$$aFlich, José$$b12
000907436 7001_ $$00000-0002-0677-6366$$aFolch, Arnau$$b13
000907436 7001_ $$00000-0002-7822-0244$$aGibbons, Steven J.$$b14
000907436 7001_ $$00000-0002-3365-8146$$aKoldunov, Nikolay$$b15
000907436 7001_ $$00000-0002-9845-8890$$aLordan, Francesc$$b16
000907436 7001_ $$00000-0002-1458-2131$$aLorito, Stefano$$b17
000907436 7001_ $$00000-0003-1019-7321$$aLøvholt, Finn$$b18
000907436 7001_ $$00000-0002-3010-8050$$aMacías, Jorge$$b19
000907436 7001_ $$00000-0001-7887-1314$$aMarozzo, Fabrizio$$b20
000907436 7001_ $$0P:(DE-HGF)0$$aMichelini, Alberto$$b21
000907436 7001_ $$00000-0003-0790-1832$$aMonterrubio-Velasco, Marisol$$b22
000907436 7001_ $$00000-0002-8207-1464$$aPienkowska, Marta$$b23
000907436 7001_ $$00000-0003-2608-1526$$ade la Puente, Josep$$b24
000907436 7001_ $$00000-0003-2782-2955$$aQueralt, Anna$$b25
000907436 7001_ $$0P:(DE-HGF)0$$aQuintana-Ortí, Enrique S.$$b26
000907436 7001_ $$00000-0002-4715-2154$$aRodríguez, Juan E.$$b27
000907436 7001_ $$00000-0003-2725-3596$$aRomano, Fabrizio$$b28
000907436 7001_ $$00000-0003-0528-7074$$aRossi, Riccardo$$b29
000907436 7001_ $$0P:(DE-Juel1)144343$$aRybicki, Jedrzej$$b30
000907436 7001_ $$0P:(DE-HGF)0$$aKupczyk, Miroslaw$$b31
000907436 7001_ $$00000-0001-6263-6934$$aSelva, Jacopo$$b32
000907436 7001_ $$0P:(DE-HGF)0$$aTalia, Domenico$$b33
000907436 7001_ $$00000-0001-7617-7206$$aTonini, Roberto$$b34
000907436 7001_ $$0P:(DE-HGF)0$$aTrunfio, Paolo$$b35
000907436 7001_ $$00000-0003-4551-3339$$aVolpe, Manuela$$b36
000907436 773__ $$0PERI:(DE-600)2020551-X$$a10.1016/j.future.2022.04.014$$gp. S0167739X22001364$$p414-429$$tFuture generation computer systems$$v134$$x0167-739X$$y2022
000907436 8564_ $$uhttps://juser.fz-juelich.de/record/907436/files/eFlows4HPC.pdf$$yPublished on 2022-04-27. Available in OpenAccess from 2024-04-27.
000907436 909CO $$ooai:juser.fz-juelich.de:907436$$pdnbdelivery$$pdriver$$pVDB$$popen_access$$popenaire
000907436 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)144343$$aForschungszentrum Jülich$$b30$$kFZJ
000907436 9131_ $$0G:(DE-HGF)POF4-511$$1G:(DE-HGF)POF4-510$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5112$$aDE-HGF$$bKey Technologies$$lEngineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action$$vEnabling Computational- & Data-Intensive Science and Engineering$$x0
000907436 9141_ $$y2022
000907436 915__ $$0LIC:(DE-HGF)CCBYNCND4$$2HGFVOC$$aCreative Commons Attribution-NonCommercial-NoDerivs CC BY-NC-ND 4.0
000907436 915__ $$0StatID:(DE-HGF)0530$$2StatID$$aEmbargoed OpenAccess
000907436 915__ $$0StatID:(DE-HGF)0113$$2StatID$$aWoS$$bScience Citation Index Expanded$$d2021-02-04
000907436 915__ $$0StatID:(DE-HGF)0160$$2StatID$$aDBCoverage$$bEssential Science Indicators$$d2021-02-04
000907436 915__ $$0StatID:(DE-HGF)0100$$2StatID$$aJCR$$bFUTURE GENER COMP SY : 2021$$d2022-11-09
000907436 915__ $$0StatID:(DE-HGF)0200$$2StatID$$aDBCoverage$$bSCOPUS$$d2022-11-09
000907436 915__ $$0StatID:(DE-HGF)0300$$2StatID$$aDBCoverage$$bMedline$$d2022-11-09
000907436 915__ $$0StatID:(DE-HGF)0199$$2StatID$$aDBCoverage$$bClarivate Analytics Master Journal List$$d2022-11-09
000907436 915__ $$0StatID:(DE-HGF)1160$$2StatID$$aDBCoverage$$bCurrent Contents - Engineering, Computing and Technology$$d2022-11-09
000907436 915__ $$0StatID:(DE-HGF)0150$$2StatID$$aDBCoverage$$bWeb of Science Core Collection$$d2022-11-09
000907436 915__ $$0StatID:(DE-HGF)9905$$2StatID$$aIF >= 5$$bFUTURE GENER COMP SY : 2021$$d2022-11-09
000907436 9201_ $$0I:(DE-Juel1)JSC-20090406$$kJSC$$lJülich Supercomputing Center$$x0
000907436 980__ $$ajournal
000907436 980__ $$aVDB
000907436 980__ $$aUNRESTRICTED
000907436 980__ $$aI:(DE-Juel1)JSC-20090406
000907436 9801_ $$aFullTexts