Contribution to a conference proceedings/Contribution to a book

FZJ-2017-08469

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Supporting Software Engineering Practices in the Development of Data-Intensive HPC Applications with the JuML Framework

Götz, M. (Corresponding author)FZJ* ; Book, M. ; Bodenstein, C.FZJ* ; Riedel, M.FZJ*

2017
ACM Press

This record in other databases:

Please use a persistent id in citations: http://hdl.handle.net/2128/26302  doi:10.1145/3144763.3144765

Abstract: The development of high performance computing applications is considerably different from traditional software development. This distinction is due to the complex hardware systems, inherent parallelism, different software lifecycle and workflow, as well as (especially for scientific computing applications) partially unknown requirements at design time. This makes the use of software engineering practices challenging, so only a small subset of them are actually applied. In this paper, we discuss the potential for applying software engineering techniques to an emerging field in high performance computing, namely large-scale data analysis and machine learning. We argue for the employment of software engineering techniques in the development of such applications from the start, and the design of generic, reusable components. Using the example of the Juelich Machine Learning Library (JuML), we demonstrate how such a framework can not only simplify the design of new parallel algorithms, but also increase the productivity of the actual data analysis workflow. We place particular focus on the abstraction from heterogeneous hardware, the architectural design as well as aspects of parallel and distributed unit testing.

---

Contributing Institute(s):

1. Jülich Supercomputing Center (JSC)

Research Program(s):

1. 512 - Data-Intensive Science and Federated Computing (POF3-512) (POF3-512)
2. PhD no Grant - Doktorand ohne besondere Förderung (PHD-NO-GRANT-20170405) (PHD-NO-GRANT-20170405)
3. DEEP-EST - DEEP - Extreme Scale Technologies (754304) (754304)

Appears in the scientific report 2017

---

Database coverage:

---