000915909 001__ 915909
000915909 005__ 20221213131511.0
000915909 0247_ $$2doi$$a10.1101/2022.08.08.503167
000915909 0247_ $$2Handle$$a2128/33051
000915909 037__ $$aFZJ-2022-05778
000915909 1001_ $$0P:(DE-HGF)0$$aChen, Jianzhong$$b0$$eCorresponding author
000915909 245__ $$aThere is no fundamental trade-off between prediction accuracy and feature importance reliability
000915909 260__ $$c2022
000915909 3367_ $$0PUB:(DE-HGF)25$$2PUB:(DE-HGF)$$aPreprint$$bpreprint$$mpreprint$$s1670908674_6832
000915909 3367_ $$2ORCID$$aWORKING_PAPER
000915909 3367_ $$028$$2EndNote$$aElectronic Article
000915909 3367_ $$2DRIVER$$apreprint
000915909 3367_ $$2BibTeX$$aARTICLE
000915909 3367_ $$2DataCite$$aOutput Types/Working Paper
000915909 520__ $$aThere is significant interest in using neuroimaging data to predict behavior. The predictive models are often interpreted by the computation of feature importance, which quantifies the predictive relevance of an imaging feature. Tian and Zalesky (2021) suggest that feature importance estimates exhibit low test-retest reliability, pointing to a potential trade-off between prediction accuracy and feature importance reliability. This trade-off is counter-intuitive because both prediction accuracy and test-retest reliability reflect the reliability of brain-behavior relationships across independent samples. Here, we revisit the relationship between prediction accuracy and feature importance reliability in a large well-powered dataset across a wide range of behavioral measures. We demonstrate that, with a sufficient sample size, feature importance (operationalized as Haufe-transformed weights) can achieve fair to excellent test-retest reliability. More specifically, with a sample size of about 2600 participants, Haufe-transformed weights achieve average intra-class correlation coefficients of 0.75, 0.57 and 0.53 for cognitive, personality and mental health measures respectively. Haufe-transformed weights are much more reliable than original regression weights and univariate FC-behavior correlations. Intriguingly, feature importance reliability is strongly positively correlated with prediction accuracy across phenotypes. Within a particular behavioral domain, there was no clear relationship between prediction performance and feature importance reliability across regression algorithms. Finally, we show mathematically that feature importance reliability is necessary, but not sufficient, for low feature importance error. In the case of linear models, lower feature importance error leads to lower prediction error (up to a scaling by the feature covariance matrix). Overall, we find no fundamental trade-off between feature importance reliability and prediction accuracy.
000915909 536__ $$0G:(DE-HGF)POF4-5251$$a5251 - Multilevel Brain Organization and Variability (POF4-525)$$cPOF4-525$$fPOF IV$$x0
000915909 588__ $$aDataset connected to CrossRef
000915909 7001_ $$0P:(DE-HGF)0$$aOoi, Leon Qi Rong$$b1
000915909 7001_ $$0P:(DE-Juel1)164828$$aLi, Jingwei$$b2
000915909 7001_ $$0P:(DE-HGF)0$$aAsplund, Christopher L.$$b3
000915909 7001_ $$0P:(DE-Juel1)131678$$aEickhoff, Simon B$$b4
000915909 7001_ $$0P:(DE-Juel1)136848$$aBzdok, Danilo$$b5
000915909 7001_ $$0P:(DE-HGF)0$$aHolmes, Avram J$$b6
000915909 7001_ $$0P:(DE-HGF)0$$aYeo, B. T. Thomas$$b7
000915909 773__ $$a10.1101/2022.08.08.503167
000915909 8564_ $$uhttps://juser.fz-juelich.de/record/915909/files/2022.08.08.503167v1.full.pdf$$yOpenAccess
000915909 909CO $$ooai:juser.fz-juelich.de:915909$$popenaire$$popen_access$$pVDB$$pdriver$$pdnbdelivery
000915909 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)164828$$aForschungszentrum Jülich$$b2$$kFZJ
000915909 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)131678$$aForschungszentrum Jülich$$b4$$kFZJ
000915909 9131_ $$0G:(DE-HGF)POF4-525$$1G:(DE-HGF)POF4-520$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5251$$aDE-HGF$$bKey Technologies$$lNatural, Artificial and Cognitive Information Processing$$vDecoding Brain Organization and Dysfunction$$x0
000915909 9141_ $$y2022
000915909 915__ $$0StatID:(DE-HGF)0510$$2StatID$$aOpenAccess
000915909 915__ $$0LIC:(DE-HGF)CCBYNCND4$$2HGFVOC$$aCreative Commons Attribution-NonCommercial-NoDerivs CC BY-NC-ND 4.0
000915909 920__ $$lyes
000915909 9201_ $$0I:(DE-Juel1)INM-7-20090406$$kINM-7$$lGehirn & Verhalten$$x0
000915909 980__ $$apreprint
000915909 980__ $$aVDB
000915909 980__ $$aUNRESTRICTED
000915909 980__ $$aI:(DE-Juel1)INM-7-20090406
000915909 9801_ $$aFullTexts