001041623 001__ 1041623
001041623 005__ 20250430202311.0
001041623 0247_ $$2doi$$a10.5194/egusphere-egu25-19091
001041623 0247_ $$2datacite_doi$$a10.34734/FZJ-2025-02352
001041623 037__ $$aFZJ-2025-02352
001041623 041__ $$aEnglish
001041623 1001_ $$0P:(DE-Juel1)171304$$aHinz, Carsten$$b0$$eCorresponding author
001041623 1112_ $$aEGU General Assembly$$cVienna$$d2025-04-28 - 2025-05-02$$gEGU25$$wAustria
001041623 245__ $$aEasier Access to ESM Data: Implementation at Jülich Supercomputing Centre
001041623 260__ $$c2025
001041623 3367_ $$033$$2EndNote$$aConference Paper
001041623 3367_ $$2BibTeX$$aINPROCEEDINGS
001041623 3367_ $$2DRIVER$$aconferenceObject
001041623 3367_ $$2ORCID$$aCONFERENCE_POSTER
001041623 3367_ $$2DataCite$$aOutput Types/Conference Poster
001041623 3367_ $$0PUB:(DE-HGF)24$$2PUB:(DE-HGF)$$aPoster$$bposter$$mposter$$s1746010978_7677$$xAfter Call
001041623 500__ $$aHow to cite according to EGU25: Hinz, C., Apweiler, S., Grasse, S., Hagemeier, B., Saini, R., and Schultz, M.: Easier Access to ESM Data: Implementation at Jülich Supercomputing Centre, EGU General Assembly 2025, Vienna, Austria, 27 Apr–2 May 2025, EGU25-19091, https://doi.org/10.5194/egusphere-egu25-19091, 2025.
001041623 520__ $$aImprovements in computational speed lead to better resolutions in Earth System Models (ESM) allowing them to resolve scales of a few kilometers. The volume of the resulting data greatly increases with the improvements in resolution and introduces a challenge to processing and storing these results.While modern HPC systems provide petabyte-scale capacity for file storage, analyzing such data on local user systems can become a prohibitive bottleneck. Beyond the sheer demands on processing high-volume ESM data, there is also increasing demand to make them FAIR and in particular findable.The goal of the “Easier” module of the Warmworld project aims to simplify the access to ESM data from different HPC centers, in particular the German Climate Computing Center (DKRZ) and the Jülich Supercomputing Centre (JSC). One aspect is the creation of a joined catalog following the SpatioTemporal Asset Catalogs (STAC) specification. This allows browsing data available at both centers. Details on the STAC implementation will be presented by Kameswar Rao Modali et al. at this General Assembly (EGU25-10288).The catalog also contains links to access the data, either as a download or later as zarr-stream. The explicit implementation of the required REST-APIs depends on the infrastructure, hardware and software, of the data centers as well as the organization of the stored data.As the first data backend at JSC, we set up a Fields DataBase (FDB), developed by ECMWF, to store the ESM results as multi-dimensional data cubes. For data retrieval, we provide a download service based on the MARS language for identifying data within the FDB and the UNiform Interface to COmputing REsources (UNICORE) for an automated access to our HPC system. The download service integrates the Helmholtz authentication and authorization infrastructure. This will allow a large number of institutions to access these services with the possibility to control access and resource use.Our poster will provide details on the ongoing implementation and combination of the various tools and services used at JSC as well as further details on the implementation on our HPC systems. In addition, we will provide first information about the planned authorization for different data accesses and also show the performance of different components with a set of benchmarks.
001041623 536__ $$0G:(DE-HGF)POF4-5111$$a5111 - Domain-Specific Simulation & Data Life Cycle Labs (SDLs) and Research Groups (POF4-511)$$cPOF4-511$$fPOF IV$$x0
001041623 536__ $$0G:(DE-HGF)POF4-5112$$a5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511)$$cPOF4-511$$fPOF IV$$x1
001041623 536__ $$0G:(DE-Juel-1)ESDE$$aEarth System Data Exploration (ESDE)$$cESDE$$x2
001041623 536__ $$0G:(DE-Juel1)BMBF-01LK2204D$$aBMBF 01LK2204D - WarmWorld Modul 3 "Easier" (BMBF-01LK2204D)$$cBMBF-01LK2204D$$x3
001041623 588__ $$aDataset connected to CrossRef
001041623 7001_ $$0P:(DE-Juel1)164455$$aApweiler, Sander$$b1
001041623 7001_ $$0P:(DE-Juel1)176624$$aGrasse, Simon$$b2
001041623 7001_ $$0P:(DE-Juel1)132123$$aHagemeier, Björn$$b3$$ufzj
001041623 7001_ $$0P:(DE-Juel1)157761$$aSaini, Rajveer$$b4$$ufzj
001041623 7001_ $$0P:(DE-Juel1)6952$$aSchultz, Martin$$b5$$eLast author
001041623 773__ $$a10.5194/egusphere-egu25-19091
001041623 8564_ $$uhttps://meetingorganizer.copernicus.org/EGU25/EGU25-19091.html
001041623 8564_ $$uhttps://juser.fz-juelich.de/record/1041623/files/EGU2025_CHinz_JSC_WWEasier_ImplementationOfServices.pdf$$yOpenAccess
001041623 909CO $$ooai:juser.fz-juelich.de:1041623$$popenaire$$popen_access$$pVDB$$pdriver
001041623 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)171304$$aForschungszentrum Jülich$$b0$$kFZJ
001041623 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)164455$$aForschungszentrum Jülich$$b1$$kFZJ
001041623 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)176624$$aForschungszentrum Jülich$$b2$$kFZJ
001041623 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)132123$$aForschungszentrum Jülich$$b3$$kFZJ
001041623 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)157761$$aForschungszentrum Jülich$$b4$$kFZJ
001041623 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)6952$$aForschungszentrum Jülich$$b5$$kFZJ
001041623 9131_ $$0G:(DE-HGF)POF4-511$$1G:(DE-HGF)POF4-510$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5111$$aDE-HGF$$bKey Technologies$$lEngineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action$$vEnabling Computational- & Data-Intensive Science and Engineering$$x0
001041623 9131_ $$0G:(DE-HGF)POF4-511$$1G:(DE-HGF)POF4-510$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5112$$aDE-HGF$$bKey Technologies$$lEngineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action$$vEnabling Computational- & Data-Intensive Science and Engineering$$x1
001041623 9141_ $$y2025
001041623 915__ $$0StatID:(DE-HGF)0510$$2StatID$$aOpenAccess
001041623 920__ $$lyes
001041623 9201_ $$0I:(DE-Juel1)JSC-20090406$$kJSC$$lJülich Supercomputing Center$$x0
001041623 980__ $$aposter
001041623 980__ $$aVDB
001041623 980__ $$aUNRESTRICTED
001041623 980__ $$aI:(DE-Juel1)JSC-20090406
001041623 9801_ $$aFullTexts