Journal Article

FZJ-2019-02426

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Big data and extreme-scale computing: Pathways to Convergence-Toward ashaping strategy for a future software and data ecosystem for scientific inquiry

Asch, M. ; Moore, T. ; Badia, R. ; Beck, M. ; Beckman, P. ; Bidot, T. ; Bodin, F. ; Cappello, F. ; Choudhary, A. ; de Supinski, B. ; Deelman, E. ; Dongarra, J. (Corresponding author) ; Dubey, A. ; Fox, G. ; Fu, H. ; Girona, S. ; Gropp, W. ; Heroux, M. ; Ishikawa, Y. ; Keahey, K. ; Keyes, D. ; Kramer, W. ; Lavignon, J.-F. ; Lu, Y. ; Matsuoka, S. ; Mohr, B.FZJ* ; Reed, D. ; Requena, S. ; Saltz, J. ; Schulthess, T. ; Stevens, R. ; Swany, M. ; Szalay, A. ; Tang, W. ; Varoquaux, G. ; Vilotte, J.-P. ; Wisniewski, R. ; Xu, Z. ; Zacharov, I.

2018
Sage Science Press Thousand Oaks, Calif.

This record in other databases:    

Please use a persistent id in citations: doi:10.1177/1094342018778123

Abstract: Over the past four years, the Big Data and Exascale Computing (BDEC) project organized a series of five international workshops that aimed to explore the ways in which the new forms of data-centric discovery introduced by the ongoing revolution in high-end data analysis (HDA) might be integrated with the established, simulation-centric paradigm of the high-performance computing (HPC) community. Based on those meetings, we argue that the rapid proliferation of digital data generators, the unprecedented growth in the volume and diversity of the data they generate, and the intense evolution of the methods for analyzing and using that data are radically reshaping the landscape of scientific computing. The most critical problems involve the logistics of wide-area, multistage workflows that will move back and forth acrossthe computing continuum, between the multitude of distributed sensors, instruments and other devices at the networks edge, and the centralized resources of commercial clouds and HPC centers. We suggest that the prospects for the future integration of technological infrastructures and research ecosystems need to be considered at three different levels. First, we discuss the convergence of research applications and workflows that establish a research paradigm that combines both HPC and HDA, where ongoing progress is already motivating efforts at the other two levels. Second, we offer an accountof some of the problems involved with creating a converged infrastructure for peripheral environments, that is, a shared infrastructure that can be deployed throughout the network in a scalable manner to meet the highly diverse requirements for processing, communication, and buffering/storage of massive data workflows of many different scientific domains. Third, we focus on some opportunities for software ecosystem convergence in big, logically centralized facilities that execute large-scale simulations and models and/or perform large-scale data analytics. We close by offering some conclusions and recommendations for future investment and policy review.

---

Contributing Institute(s):

1. Jülich Supercomputing Center (JSC)

Research Program(s):

1. 511 - Computational Science and Mathematical Methods (POF3-511) (POF3-511)

Appears in the scientific report 2019

---

Database coverage:
; Allianz-Lizenz ; Clarivate Analytics Master Journal List ; Current Contents - Engineering, Computing and Technology ; Ebsco Academic Search ; IF < 5 ; JCR ; Nationallizenz ; SCOPUS ; Science Citation Index ; Science Citation Index Expanded ; Web of Science Core Collection

---