001006999 001__ 1006999
001006999 005__ 20231027114402.0
001006999 0247_ $$2doi$$a10.1016/j.neuroimage.2023.120115
001006999 0247_ $$2ISSN$$a1053-8119
001006999 0247_ $$2ISSN$$a1095-9572
001006999 0247_ $$2Handle$$a2128/34436
001006999 0247_ $$2pmid$$a37088322
001006999 0247_ $$2WOS$$aWOS:001005138000001
001006999 037__ $$aFZJ-2023-01938
001006999 082__ $$a610
001006999 1001_ $$0P:(DE-HGF)0$$aChen, Jianzhong$$b0
001006999 245__ $$aRelationship Between Prediction Accuracy and Feature Importance Reliability: an Empirical and Theoretical Study
001006999 260__ $$aOrlando, Fla.$$bAcademic Press$$c2023
001006999 3367_ $$2DRIVER$$aarticle
001006999 3367_ $$2DataCite$$aOutput Types/Journal article
001006999 3367_ $$0PUB:(DE-HGF)16$$2PUB:(DE-HGF)$$aJournal Article$$bjournal$$mjournal$$s1684227859_22304
001006999 3367_ $$2BibTeX$$aARTICLE
001006999 3367_ $$2ORCID$$aJOURNAL_ARTICLE
001006999 3367_ $$00$$2EndNote$$aJournal Article
001006999 520__ $$aThere is significant interest in using neuroimaging data to predict behavior. The predictive models are often interpreted by the computation of feature importance, which quantifies the predictive relevance of an imaging feature. Tian and Zalesky (2021) suggest that feature importance estimates exhibit low split-half reliability, as well as a trade-off between prediction accuracy and feature importance reliability across parcellation resolutions. However, it is unclear whether the trade-off between prediction accuracy and feature importance reliability is universal. Here, we demonstrate that, with a sufficient sample size, feature importance (operationalized as Haufe-transformed weights) can achieve fair to excellent split-half reliability. With a sample size of 2600 participants, Haufe-transformed weights achieve average intra-class correlation coefficients of 0.75, 0.57 and 0.53 for cognitive, personality and mental health measures respectively. Haufe-transformed weights are much more reliable than original regression weights and univariate FC-behavior correlations. Original regression weights are not reliable even with 2600 participants. Intriguingly, feature importance reliability is strongly positively correlated with prediction accuracy across phenotypes. Within a particular behavioral domain, there is no clear relationship between prediction performance and feature importance reliability across regression models. Furthermore, we show mathematically that feature importance reliability is necessary, but not sufficient, for low feature importance error. In the case of linear models, lower feature importance error is mathematically related to lower prediction error. Therefore, higher feature importance reliability might yield lower feature importance error and higher prediction accuracy. Finally, we discuss how our theoretical results relate with the reliability of imaging features and behavioral measures. Overall, the current study provides empirical and theoretical insights into the relationship between prediction accuracy and feature importance reliability.
001006999 536__ $$0G:(DE-HGF)POF4-5251$$a5251 - Multilevel Brain Organization and Variability (POF4-525)$$cPOF4-525$$fPOF IV$$x0
001006999 588__ $$aDataset connected to CrossRef, Journals: juser.fz-juelich.de
001006999 7001_ $$0P:(DE-HGF)0$$aOoi, Leon Qi Rong$$b1
001006999 7001_ $$0P:(DE-HGF)0$$aTan, Trevor Wei Kiat$$b2
001006999 7001_ $$0P:(DE-HGF)0$$aZhang, Shaoshi$$b3
001006999 7001_ $$0P:(DE-Juel1)164828$$aLi, Jingwei$$b4
001006999 7001_ $$0P:(DE-HGF)0$$aAsplund, Christopher L.$$b5
001006999 7001_ $$0P:(DE-Juel1)131678$$aEickhoff, Simon B$$b6
001006999 7001_ $$0P:(DE-HGF)0$$aBzdok, Danilo$$b7
001006999 7001_ $$0P:(DE-HGF)0$$aHolmes, Avram J$$b8
001006999 7001_ $$0P:(DE-HGF)0$$aYeo, B. T. Thomas$$b9$$eCorresponding author
001006999 773__ $$0PERI:(DE-600)1471418-8$$a10.1016/j.neuroimage.2023.120115$$gp. 120115 -$$p120115 -$$tNeuroImage$$v274$$x1053-8119$$y2023
001006999 8564_ $$uhttps://juser.fz-juelich.de/record/1006999/files/1-s2.0-S1053811923002616-main.pdf$$yOpenAccess
001006999 909CO $$ooai:juser.fz-juelich.de:1006999$$pdnbdelivery$$pdriver$$pVDB$$popen_access$$popenaire
001006999 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)164828$$aForschungszentrum Jülich$$b4$$kFZJ
001006999 9101_ $$0I:(DE-HGF)0$$6P:(DE-Juel1)164828$$a HHU Düsseldorf$$b4
001006999 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)131678$$aForschungszentrum Jülich$$b6$$kFZJ
001006999 9101_ $$0I:(DE-HGF)0$$6P:(DE-Juel1)131678$$a HHU Düsseldorf$$b6
001006999 9101_ $$0I:(DE-HGF)0$$6P:(DE-HGF)0$$a Centre for Sleep and Cognition, Yong Loo Lin School of Medicine, National University of Singapore$$b9
001006999 9131_ $$0G:(DE-HGF)POF4-525$$1G:(DE-HGF)POF4-520$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5251$$aDE-HGF$$bKey Technologies$$lNatural, Artificial and Cognitive Information Processing$$vDecoding Brain Organization and Dysfunction$$x0
001006999 9141_ $$y2023
001006999 915__ $$0StatID:(DE-HGF)0160$$2StatID$$aDBCoverage$$bEssential Science Indicators$$d2022-11-12
001006999 915__ $$0StatID:(DE-HGF)1190$$2StatID$$aDBCoverage$$bBiological Abstracts$$d2022-11-12
001006999 915__ $$0LIC:(DE-HGF)CCBYNCND4$$2HGFVOC$$aCreative Commons Attribution-NonCommercial-NoDerivs CC BY-NC-ND 4.0
001006999 915__ $$0StatID:(DE-HGF)0113$$2StatID$$aWoS$$bScience Citation Index Expanded$$d2022-11-12
001006999 915__ $$0StatID:(DE-HGF)0700$$2StatID$$aFees$$d2022-11-12
001006999 915__ $$0StatID:(DE-HGF)0510$$2StatID$$aOpenAccess
001006999 915__ $$0StatID:(DE-HGF)0561$$2StatID$$aArticle Processing Charges$$d2022-11-12
001006999 915__ $$0StatID:(DE-HGF)0501$$2StatID$$aDBCoverage$$bDOAJ Seal$$d2023-05-02T08:47:40Z
001006999 915__ $$0StatID:(DE-HGF)0500$$2StatID$$aDBCoverage$$bDOAJ$$d2023-05-02T08:47:40Z
001006999 915__ $$0StatID:(DE-HGF)0030$$2StatID$$aPeer Review$$bDOAJ : Anonymous peer review$$d2023-05-02T08:47:40Z
001006999 915__ $$0StatID:(DE-HGF)0420$$2StatID$$aNationallizenz$$d2023-10-21$$wger
001006999 915__ $$0StatID:(DE-HGF)0100$$2StatID$$aJCR$$bNEUROIMAGE : 2022$$d2023-10-21
001006999 915__ $$0StatID:(DE-HGF)0200$$2StatID$$aDBCoverage$$bSCOPUS$$d2023-10-21
001006999 915__ $$0StatID:(DE-HGF)0300$$2StatID$$aDBCoverage$$bMedline$$d2023-10-21
001006999 915__ $$0StatID:(DE-HGF)0600$$2StatID$$aDBCoverage$$bEbsco Academic Search$$d2023-10-21
001006999 915__ $$0StatID:(DE-HGF)0030$$2StatID$$aPeer Review$$bASC$$d2023-10-21
001006999 915__ $$0StatID:(DE-HGF)0199$$2StatID$$aDBCoverage$$bClarivate Analytics Master Journal List$$d2023-10-21
001006999 915__ $$0StatID:(DE-HGF)1050$$2StatID$$aDBCoverage$$bBIOSIS Previews$$d2023-10-21
001006999 915__ $$0StatID:(DE-HGF)0150$$2StatID$$aDBCoverage$$bWeb of Science Core Collection$$d2023-10-21
001006999 915__ $$0StatID:(DE-HGF)1030$$2StatID$$aDBCoverage$$bCurrent Contents - Life Sciences$$d2023-10-21
001006999 915__ $$0StatID:(DE-HGF)9905$$2StatID$$aIF >= 5$$bNEUROIMAGE : 2022$$d2023-10-21
001006999 920__ $$lyes
001006999 9201_ $$0I:(DE-Juel1)INM-7-20090406$$kINM-7$$lGehirn & Verhalten$$x0
001006999 980__ $$ajournal
001006999 980__ $$aVDB
001006999 980__ $$aUNRESTRICTED
001006999 980__ $$aI:(DE-Juel1)INM-7-20090406
001006999 9801_ $$aFullTexts