Conference Presentation (After Call)

FZJ-2024-00088

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Learning from the present for the future:The Jülich LOFAR Long-Term Archive

Stiele, H. (Corresponding author)FZJ* ; Manzano, C.FZJ* ; Miskolczi, A.FZJ* ; Vybornov, V.FZJ* ; Pfalzner, S.FZJ* ; Fieseler, T.FZJ*

2023

Abstract: Forschungszentrum Jülich has hosted the German part of the LOFAR long-term archive since 2013. Currently about 20 petabytes (PB) of data are stored with a growth rate of around 2 PB a year.Future radio telescopes are expected to have a much higher data rate and bring new challenges in processing and storing data.Here we briefly report on the current data management of the Jülich LOFAR Data Archive, including the ingestion, the storage system, the export to the long-term archive, and the request chain. We analysed the data access pattern over the last 10 years and give an estimation of the energy consumption of the process. Based on this analysis, we define requirements for future, even more extensive long-term data archives.

---

Contributing Institute(s):

1. Jülich Supercomputing Center (JSC)

Research Program(s):

1. 5111 - Domain-Specific Simulation & Data Life Cycle Labs (SDLs) and Research Groups (POF4-511) (POF4-511)
2. 5121 - Supercomputing & Big Data Facilities (POF4-512) (POF4-512)
3. DFG project 460248186 - PUNCH4NFDI - Teilchen, Universum, Kerne und Hadronen für die NFDI (460248186) (460248186)
4. Big Bang to Big Data - B3D [NRW-Cluster für datenintensive Radioastronomie] (PROFILNRW-2020-038B) (PROFILNRW-2020-038B)

Appears in the scientific report 2023

---