Journal Article

FZJ-2024-01272

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png METhodological RadiomICs Score (METRICS): a quality scoring tool for radiomics research endorsed by EuSoMII

Kocak, B. ; Akinci D’Antonoli, T. (Corresponding author) ; Mercaldo, N. ; Alberich-Bayarri, A. ; Baessler, B. ; Ambrosini, I. ; Andreychenko, A. E. ; Bakas, S. ; Beets-Tan, R. G. H. ; Bressem, K. ; Buvat, I. ; Cannella, R. ; Cappellini, L. A. ; Cavallo, A. U. ; Chepelev, L. L. ; Chu, L. C. H. ; Demircioglu, A. ; deSouza, N. M. ; Dietzel, M. ; Fanni, S. C. ; Fedorov, A. ; Fournier, L. S. ; Giannini, V. ; Girometti, R. ; Groot Lipman, K. B. W. ; Kalarakis, G. ; Kelly, B. S. ; Klontzas, M. E. ; Koh, D.-M. ; Kotter, E. ; Lee, H. Y. ; Maas, M. ; Marti-Bonmati, L. ; Müller, H. ; Obuchowski, N. ; Orlhac, F. ; Papanikolaou, N. ; Petrash, E. ; Pfaehler, E.FZJ* ; Pinto dos Santos, D. ; Ponsiglione, A. ; Sabater, S. ; Sardanelli, F. ; Seeböck, P. ; Sijtsema, N. M. ; Stanzione, A. ; Traverso, A. ; Ugga, L. ; Vallières, M. ; van Dijk, L. V. ; van Griethuysen, J. J. M. ; van Hamersvelt, R. W. ; van Ooijen, P. ; Vernuccio, F. ; Wang, A. ; Williams, S. ; Witowski, J. ; Zhang, Z. ; Zwanenburg, A. ; Cuocolo, R.

2024
Springer Heidelberg

This record in other databases:      

Please use a persistent id in citations: doi:10.1186/s13244-023-01572-w  doi:10.34734/FZJ-2024-01272

Abstract: Purpose: To propose a new quality scoring tool, METhodological RadiomICs Score (METRICS), to assess and improve research quality of radiomics studies.Methods: We conducted an online modified Delphi study with a group of international experts. It was performed in three consecutive stages: Stage#1, item preparation; Stage#2, panel discussion among EuSoMII Auditing Group members to identify the items to be voted; and Stage#3, four rounds of the modified Delphi exercise by panelists to determine the items eligible for the METRICS and their weights. The consensus threshold was 75%. Based on the median ranks derived from expert panel opinion and their rank-sum based conversion to importance scores, the category and item weights were calculated.Result: In total, 59 panelists from 19 countries participated in selection and ranking of the items and categories. Final METRICS tool included 30 items within 9 categories. According to their weights, the categories were in descending order of importance: study design, imaging data, image processing and feature extraction, metrics and comparison, testing, feature processing, preparation for modeling, segmentation, and open science. A web application and a repository were developed to streamline the calculation of the METRICS score and to collect feedback from the radiomics community.Conclusion: In this work, we developed a scoring tool for assessing the methodological quality of the radiomics research, with a large international panel and a modified Delphi protocol. With its conditional format to cover methodological variations, it provides a well-constructed framework for the key methodological concepts to assess the quality of radiomic research papers.Critical relevance statement: A quality assessment tool, METhodological RadiomICs Score (METRICS), is made available by a large group of international domain experts, with transparent methodology, aiming at evaluating and improving research quality in radiomics and machine learning.Key points:• A methodological scoring tool, METRICS, was developed for assessing the quality of radiomics research, with a large international expert panel and a modified Delphi protocol.• The proposed scoring tool presents expert opinion-based importance weights of categories and items with a transparent methodology for the first time.• METRICS accounts for varying use cases, from handcrafted radiomics to entirely deep learning-based pipelines.• A web application has been developed to help with the calculation of the METRICS score (https://metricsscore.github.io/metrics/METRICS.html) and a repository created to collect feedback from the radiomics community (https://github.com/metricsscore/metrics).

---

Contributing Institute(s):

1. Datenanalyse und Maschinenlernen (IAS-8)

Research Program(s):

1. 5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511) (POF4-511)
2. 510 - Engineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action (POF4-500) (POF4-500)

Appears in the scientific report 2024

---

Database coverage:
; ; ; ; Clarivate Analytics Master Journal List ; Current Contents - Clinical Medicine ; DOAJ Seal ; Ebsco Academic Search ; IF < 5 ; JCR ; SCOPUS ; Web of Science Core Collection

---