Conference Presentation (After Call)

FZJ-2019-03958

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png A New Tool for Automated Quality Control of Environmental Time Series (AutoQC4Env) in Open Web Services

Kaffashzadeh, N. (Corresponding author)FZJ* ; Kleinert, F.FZJ* ; Schultz, M.FZJ*

2019

Please use a persistent id in citations: http://hdl.handle.net/2128/22565

Abstract: We report on the development of a new software tool (AutoQC4Env) for automated quality control (QC) of environmental timeseries data. Novel features of this tool include a flexible Python softwarearchitecture, which makes it easy for users to configure the sequence oftests as well as their statistical parameters, and a statistical concept toassign each value a probability of being a valid data point. There aremany occasions when it is necessary to inspect the quality of environmental data sets, from first quality checks during real-time sampling anddata transmission to assessing the quality and consistency of long-termmonitoring data from measurement stations. Erroneous data can havea substantial impact on the statistical data analysis and, for example,lead to wrong estimates of trends. Existing QC workflows largely rely onindividual investigator knowledge and have been constructed from practical considerations and with a least theoretical foundation. The statistical framework that is being developed in AutoQC4Env aims to complement traditional data quality assessments and provide environmentalresearchers with a tool that is easy to use but also based on currentstatistical knowledge.

---

Contributing Institute(s):

1. Jülich Supercomputing Center (JSC)

Research Program(s):

1. 512 - Data-Intensive Science and Federated Computing (POF3-512) (POF3-512)
2. IntelliAQ - Artificial Intelligence for Air Quality (787576) (787576)
3. Earth System Data Exploration (ESDE) (ESDE)

Appears in the scientific report 2019

---

Database coverage:

---