000875320 001__ 875320
000875320 005__ 20230711152806.0
000875320 0247_ $$2doi$$a10.5194/egusphere-egu2020-13637
000875320 0247_ $$2Handle$$a2128/24883
000875320 037__ $$aFZJ-2020-01951
000875320 041__ $$aEnglish
000875320 1001_ $$0P:(DE-Juel1)171435$$aBetancourt, Clara$$b0$$eCorresponding author$$ufzj
000875320 1112_ $$aEGU2020: Sharing Geoscience Online$$c$$d2020-05-04 - 2020-05-08$$gEGU2020$$w
000875320 245__ $$aPerformance analysis and optimization of a TByte-scale atmospheric observation database
000875320 260__ $$c2020
000875320 3367_ $$0PUB:(DE-HGF)1$$2PUB:(DE-HGF)$$aAbstract$$babstract$$mabstract$$s1595508163_6250
000875320 3367_ $$033$$2EndNote$$aConference Paper
000875320 3367_ $$2BibTeX$$aINPROCEEDINGS
000875320 3367_ $$2DRIVER$$aconferenceObject
000875320 3367_ $$2DataCite$$aOutput Types/Conference Abstract
000875320 3367_ $$2ORCID$$aOTHER
000875320 520__ $$a<p>The Tropospheric Ozone Assessment Report (TOAR) created one of the world&#8217;s largest databases for near-surface air quality measurements. More than 150 users from 35 countries have accessed TOAR data via a graphical web interface (https://join.fz-juelich.de) or a REST API (https://join.fz-juelich.de/services/rest/surfacedata/) and downloaded station information and aggregated statistics of ozone and associated variables. All statistics are calculated online from the hourly data that are stored in the database to allow for maximum user flexibility (it is possible, for example, to specify the minimum data capture criterion that shall be used in the aggregation). Thus, it is of paramount importance to measure and, if necessary, optimize the performance of the database and of the web services, which are connected to it. In this work, two aspects of the TOAR database service infrastructure are investigated: Performance enhancements by database tuning and the implementation of flux-based ozone metrics, which &#8211; unlike the already existing concentration based metrics &#8211; require meteorological data and embedded modeling.</p><p>The TOAR database is a PostgreSQL V10 relational database hosted on a virtual machine, connected to the JOIN web server. In the current set-up the web services trigger SQL queries and the resulting raw data are transferred on demand to the JOIN server and processed locally to derive the requested statistical quantities. We tested the following measures to increase the database performance: optimal definition of indices, server-side programming in PL/pgSQL and PL/Python, on-line aggregation to avoid transfer of large data, and query enhancement by the explain-analyze tool of PostgreSQL. Through a combination of the above mentioned techniques, the performance of JOIN can be improved in a range of 20 - 70 %.</p><p>Flux-based ozone metrics are necessary for an accurate quantification of ozone damage on vegetation. In contrast to the already available concentration based metrics of ozone, they require the input of meteorological and soil data, as well as a consistent parametrization of vegetation growing seasons and the inclusion of a stomatal flux model. Embedding this model with the TOAR database will make a global assessment of stomatal ozone fluxes possible for the first time ever. This requires new query patterns, which need to merge several variables onto a consistent time axis, as well as more elaborate calculations, which are presently coded in FORTRAN.</p><p>The presentation will present the results from the performance tuning and discuss the pros and cons of various ways how the ozone flux calculations can be implemented.</p>
000875320 536__ $$0G:(DE-HGF)POF3-512$$a512 - Data-Intensive Science and Federated Computing (POF3-512)$$cPOF3-512$$fPOF III$$x0
000875320 536__ $$0G:(EU-Grant)787576$$aIntelliAQ - Artificial Intelligence for Air Quality (787576)$$c787576$$fERC-2017-ADG$$x1
000875320 536__ $$0G:(DE-Juel1)PHD-NO-GRANT-20170405$$aPhD no Grant - Doktorand ohne besondere Förderung (PHD-NO-GRANT-20170405)$$cPHD-NO-GRANT-20170405$$x2
000875320 536__ $$0G:(DE-Juel-1)ESDE$$aEarth System Data Exploration (ESDE)$$cESDE$$x3
000875320 588__ $$aDataset connected to CrossRef
000875320 7001_ $$0P:(DE-Juel1)16212$$aSchröder, Sabine$$b1$$ufzj
000875320 7001_ $$0P:(DE-Juel1)132123$$aHagemeier, Björn$$b2$$ufzj
000875320 7001_ $$0P:(DE-Juel1)6952$$aSchultz, Martin$$b3
000875320 773__ $$a10.5194/egusphere-egu2020-13637
000875320 8564_ $$uhttps://meetingorganizer.copernicus.org/EGU2020/EGU2020-13637.html
000875320 8564_ $$uhttps://juser.fz-juelich.de/record/875320/files/betancourt_EGU2020.pdf$$yOpenAccess
000875320 8564_ $$uhttps://juser.fz-juelich.de/record/875320/files/betancourt_EGU2020.pdf?subformat=pdfa$$xpdfa$$yOpenAccess
000875320 909CO $$ooai:juser.fz-juelich.de:875320$$pec_fundedresources$$pdriver$$pVDB$$popen_access$$popenaire
000875320 915__ $$0StatID:(DE-HGF)0510$$2StatID$$aOpenAccess
000875320 915__ $$0LIC:(DE-HGF)CCBY4$$2HGFVOC$$aCreative Commons Attribution CC BY 4.0
000875320 9141_ $$y2020
000875320 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)171435$$aForschungszentrum Jülich$$b0$$kFZJ
000875320 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)16212$$aForschungszentrum Jülich$$b1$$kFZJ
000875320 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)132123$$aForschungszentrum Jülich$$b2$$kFZJ
000875320 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)6952$$aForschungszentrum Jülich$$b3$$kFZJ
000875320 9131_ $$0G:(DE-HGF)POF3-512$$1G:(DE-HGF)POF3-510$$2G:(DE-HGF)POF3-500$$3G:(DE-HGF)POF3$$4G:(DE-HGF)POF$$aDE-HGF$$bKey Technologies$$lSupercomputing & Big Data$$vData-Intensive Science and Federated Computing$$x0
000875320 920__ $$lyes
000875320 9201_ $$0I:(DE-Juel1)JSC-20090406$$kJSC$$lJülich Supercomputing Center$$x0
000875320 980__ $$aabstract
000875320 980__ $$aVDB
000875320 980__ $$aI:(DE-Juel1)JSC-20090406
000875320 980__ $$aUNRESTRICTED
000875320 980__ $$aOPENSCIENCE
000875320 9801_ $$aFullTexts