Journal Article

FZJ-2025-04636

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png The Influence of Human Annotation on CNN Performance for Anomaly Detection in ICU Data

Goldermann, L. (Corresponding author) ; Fonck, S. ; Olivier, L. ; Fritsch, S.FZJ* ; Stollenwerk, A.

2025
De Gruyter Berlin

This record in other databases:  

Please use a persistent id in citations: doi:10.1515/cdbme-2025-0192  doi:10.34734/FZJ-2025-04636

Abstract: Deep learning methods are increasingly used in clinical artificial intelligence (AI) research, including for detecting anomalies in intensive care data. However, their evaluation often depends on human annotations, which can vary in quality and consistency. In this study, we analyse the effect of annotation variability on the performance of DeepAnT, an unsupervised convolutional neural network for anomaly detection (AD). Using intensive care time-series data from 38 patients for training and six patients separately annotated for evaluation, we compare F1 scores based on two independent physician annotations. Our results show differences in model performance across different vital parameters, between patients, and especially between annotators evaluating the same data. These findings indicate that human labelling has a measurable impact on the perceived performance of the AD algorithm. Structured labelling protocols may be beneficial for achieving more consistent and reliable evaluations.

---

Contributing Institute(s):

1. Jülich Supercomputing Center (JSC)
2. Center for Advanced Simulation and Analytics (CASA)

Research Program(s):

1. 5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511) (POF4-511)

Appears in the scientific report 2025

---

Database coverage:
; ; ; ; Article Processing Charges ; DOAJ Seal ; Fees ; SCOPUS

---