Journal Article

FZJ-2021-02653

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Context aware benchmarking and tuning of a TByte-scale air quality database and web service

Betancourt, C. (Corresponding author)FZJ* ; Hagemeier, B.FZJ* ; Schröder, S.FZJ* ; Schultz, M. G.FZJ*

2021
Springer Heidelberg

This record in other databases:      

Please use a persistent id in citations: http://hdl.handle.net/2128/28504  doi:10.1007/s12145-021-00631-4

Abstract: We present context-aware benchmarking and performance engineering of a mature TByte-scale air quality database system which was created by the Tropospheric Ozone Assessment Report (TOAR) and contains one of the world’s largest collections of near-surface air quality measurements. A special feature of our data service https://join.fz-juelich.de is on-demand processing of several air quality metrics directly from the TOAR database. As a service that is used by more than 350 users of the international air quality research community, our web service must be easily accessible and functionally flexible, while delivering good performance. The current on-demand calculations of air quality metrics outside the database together with the necessary transfer of large volume raw data are identified as the major performance bottleneck. In this study, we therefore explore and benchmark in-database approaches for the statistical processing, which results in performance enhancements of up to 32%.

---

Contributing Institute(s):

1. Jülich Supercomputing Center (JSC)
2. John von Neumann - Institut für Computing (NIC)

Research Program(s):

1. 511 - Enabling Computational- & Data-Intensive Science and Engineering (POF4-511) (POF4-511)
2. IntelliAQ - Artificial Intelligence for Air Quality (787576) (787576)
3. Deep Learning for Air Quality and Climate Forecasts (deepacf_20191101) (deepacf_20191101)
4. Earth System Data Exploration (ESDE) (ESDE)

Appears in the scientific report 2021

---

Database coverage:
; ; ; Clarivate Analytics Master Journal List ; DEAL Springer ; Ebsco Academic Search ; Essential Science Indicators ; IF < 5 ; JCR ; SCOPUS ; Science Citation Index Expanded ; Web of Science Core Collection

---