%0 Journal Article
%A Kocak, Burak
%A Akinci D’Antonoli, Tugba
%A Mercaldo, Nathaniel
%A Alberich-Bayarri, Angel
%A Baessler, Bettina
%A Ambrosini, Ilaria
%A Andreychenko, Anna E.
%A Bakas, Spyridon
%A Beets-Tan, Regina G. H.
%A Bressem, Keno
%A Buvat, Irene
%A Cannella, Roberto
%A Cappellini, Luca Alessandro
%A Cavallo, Armando Ugo
%A Chepelev, Leonid L.
%A Chu, Linda Chi Hang
%A Demircioglu, Aydin
%A deSouza, Nandita M.
%A Dietzel, Matthias
%A Fanni, Salvatore Claudio
%A Fedorov, Andrey
%A Fournier, Laure S.
%A Giannini, Valentina
%A Girometti, Rossano
%A Groot Lipman, Kevin B. W.
%A Kalarakis, Georgios
%A Kelly, Brendan S.
%A Klontzas, Michail E.
%A Koh, Dow-Mu
%A Kotter, Elmar
%A Lee, Ho Yun
%A Maas, Mario
%A Marti-Bonmati, Luis
%A Müller, Henning
%A Obuchowski, Nancy
%A Orlhac, Fanny
%A Papanikolaou, Nikolaos
%A Petrash, Ekaterina
%A Pfaehler, Elisabeth
%A Pinto dos Santos, Daniel
%A Ponsiglione, Andrea
%A Sabater, Sebastià
%A Sardanelli, Francesco
%A Seeböck, Philipp
%A Sijtsema, Nanna M.
%A Stanzione, Arnaldo
%A Traverso, Alberto
%A Ugga, Lorenzo
%A Vallières, Martin
%A van Dijk, Lisanne V.
%A van Griethuysen, Joost J. M.
%A van Hamersvelt, Robbert W.
%A van Ooijen, Peter
%A Vernuccio, Federica
%A Wang, Alan
%A Williams, Stuart
%A Witowski, Jan
%A Zhang, Zhongyi
%A Zwanenburg, Alex
%A Cuocolo, Renato
%T METhodological RadiomICs Score (METRICS): a quality scoring tool for radiomics research endorsed by EuSoMII
%J Insights into imaging
%V 15
%N 1
%@ 1869-4101
%C Heidelberg
%I Springer
%M FZJ-2024-01272
%P 8
%D 2024
%X Purpose: To propose a new quality scoring tool, METhodological RadiomICs Score (METRICS), to assess and improve research quality of radiomics studies.Methods: We conducted an online modified Delphi study with a group of international experts. It was performed in three consecutive stages: Stage#1, item preparation; Stage#2, panel discussion among EuSoMII Auditing Group members to identify the items to be voted; and Stage#3, four rounds of the modified Delphi exercise by panelists to determine the items eligible for the METRICS and their weights. The consensus threshold was 75%. Based on the median ranks derived from expert panel opinion and their rank-sum based conversion to importance scores, the category and item weights were calculated.Result: In total, 59 panelists from 19 countries participated in selection and ranking of the items and categories. Final METRICS tool included 30 items within 9 categories. According to their weights, the categories were in descending order of importance: study design, imaging data, image processing and feature extraction, metrics and comparison, testing, feature processing, preparation for modeling, segmentation, and open science. A web application and a repository were developed to streamline the calculation of the METRICS score and to collect feedback from the radiomics community.Conclusion: In this work, we developed a scoring tool for assessing the methodological quality of the radiomics research, with a large international panel and a modified Delphi protocol. With its conditional format to cover methodological variations, it provides a well-constructed framework for the key methodological concepts to assess the quality of radiomic research papers.Critical relevance statement: A quality assessment tool, METhodological RadiomICs Score (METRICS), is made available by a large group of international domain experts, with transparent methodology, aiming at evaluating and improving research quality in radiomics and machine learning.Key points:• A methodological scoring tool, METRICS, was developed for assessing the quality of radiomics research, with a large international expert panel and a modified Delphi protocol.• The proposed scoring tool presents expert opinion-based importance weights of categories and items with a transparent methodology for the first time.• METRICS accounts for varying use cases, from handcrafted radiomics to entirely deep learning-based pipelines.• A web application has been developed to help with the calculation of the METRICS score (https://metricsscore.github.io/metrics/METRICS.html) and a repository created to collect feedback from the radiomics community (https://github.com/metricsscore/metrics).
%F PUB:(DE-HGF)16
%9 Journal Article
%$ 38228979
%U <Go to ISI:>//WOS:001143356000001
%R 10.1186/s13244-023-01572-w
%U https://juser.fz-juelich.de/record/1022151