Contribution to a book

FZJ-2024-06183

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Storage and Long Term Archives

Manzano, C. (Corresponding author)FZJ* ; Schwarz, K. ; Grange, Y. G. ; Miskolczi, A.FZJ* ; Huynh, M. T. ; Wicenec, A. ; Salgado, J. ; Desai, V.

2024
Springer International Publishing Cham
ISBN: 978-3-031-58467-1, 978-3-031-58468-8 (electronic)

This record in other databases:

Please use a persistent id in citations: doi:10.1007/978-3-031-58468-8_3

Abstract: In this chapter we introduce and describe the storage and long-term archives used for radio astronomy data. Firstly, we give an overview of a storage system and explain what caching is and why it is used. Secondly, we describe in detail current solutions for data from the LOFAR, ASKAP, Apertif and ALMA radio telescopes. We then explain which data transfer protocols are commonly used and describe them. Thirdly, we present federated storage infrastructures in high-energy physics and show how they are designed. Data lakes are discussed and several projects dealing with them are presented and described. We then discuss what data lakes are or how they can be used in radio astronomy and the overlap between the data needs of astronomical archives and those of high-energy physics experiments. Finally, we give an overview of how the SKA data centres are planned.

---

Contributing Institute(s):

1. Jülich Supercomputing Center (JSC)

Research Program(s):

1. 5121 - Supercomputing & Big Data Facilities (POF4-512) (POF4-512)

Appears in the scientific report 2024

---