guest ::
| Home > Publications database > Relationship Between Prediction Accuracy and Feature Importance Reliability: an Empirical and Theoretical Study > print |
| Home > Publications database > Relationship Between Prediction Accuracy and Feature Importance Reliability: an Empirical and Theoretical Study > print |
| 001 | 1006999 | ||
| 005 | 20231027114402.0 | ||
| 024 | 7 | _ | |a 10.1016/j.neuroimage.2023.120115 |2 doi |
| 024 | 7 | _ | |a 1053-8119 |2 ISSN |
| 024 | 7 | _ | |a 1095-9572 |2 ISSN |
| 024 | 7 | _ | |a 2128/34436 |2 Handle |
| 024 | 7 | _ | |a 37088322 |2 pmid |
| 024 | 7 | _ | |a WOS:001005138000001 |2 WOS |
| 037 | _ | _ | |a FZJ-2023-01938 |
| 082 | _ | _ | |a 610 |
| 100 | 1 | _ | |a Chen, Jianzhong |0 P:(DE-HGF)0 |b 0 |
| 245 | _ | _ | |a Relationship Between Prediction Accuracy and Feature Importance Reliability: an Empirical and Theoretical Study |
| 260 | _ | _ | |a Orlando, Fla. |c 2023 |b Academic Press |
| 336 | 7 | _ | |a article |2 DRIVER |
| 336 | 7 | _ | |a Output Types/Journal article |2 DataCite |
| 336 | 7 | _ | |a Journal Article |b journal |m journal |0 PUB:(DE-HGF)16 |s 1684227859_22304 |2 PUB:(DE-HGF) |
| 336 | 7 | _ | |a ARTICLE |2 BibTeX |
| 336 | 7 | _ | |a JOURNAL_ARTICLE |2 ORCID |
| 336 | 7 | _ | |a Journal Article |0 0 |2 EndNote |
| 520 | _ | _ | |a There is significant interest in using neuroimaging data to predict behavior. The predictive models are often interpreted by the computation of feature importance, which quantifies the predictive relevance of an imaging feature. Tian and Zalesky (2021) suggest that feature importance estimates exhibit low split-half reliability, as well as a trade-off between prediction accuracy and feature importance reliability across parcellation resolutions. However, it is unclear whether the trade-off between prediction accuracy and feature importance reliability is universal. Here, we demonstrate that, with a sufficient sample size, feature importance (operationalized as Haufe-transformed weights) can achieve fair to excellent split-half reliability. With a sample size of 2600 participants, Haufe-transformed weights achieve average intra-class correlation coefficients of 0.75, 0.57 and 0.53 for cognitive, personality and mental health measures respectively. Haufe-transformed weights are much more reliable than original regression weights and univariate FC-behavior correlations. Original regression weights are not reliable even with 2600 participants. Intriguingly, feature importance reliability is strongly positively correlated with prediction accuracy across phenotypes. Within a particular behavioral domain, there is no clear relationship between prediction performance and feature importance reliability across regression models. Furthermore, we show mathematically that feature importance reliability is necessary, but not sufficient, for low feature importance error. In the case of linear models, lower feature importance error is mathematically related to lower prediction error. Therefore, higher feature importance reliability might yield lower feature importance error and higher prediction accuracy. Finally, we discuss how our theoretical results relate with the reliability of imaging features and behavioral measures. Overall, the current study provides empirical and theoretical insights into the relationship between prediction accuracy and feature importance reliability. |
| 536 | _ | _ | |a 5251 - Multilevel Brain Organization and Variability (POF4-525) |0 G:(DE-HGF)POF4-5251 |c POF4-525 |f POF IV |x 0 |
| 588 | _ | _ | |a Dataset connected to CrossRef, Journals: juser.fz-juelich.de |
| 700 | 1 | _ | |a Ooi, Leon Qi Rong |0 P:(DE-HGF)0 |b 1 |
| 700 | 1 | _ | |a Tan, Trevor Wei Kiat |0 P:(DE-HGF)0 |b 2 |
| 700 | 1 | _ | |a Zhang, Shaoshi |0 P:(DE-HGF)0 |b 3 |
| 700 | 1 | _ | |a Li, Jingwei |0 P:(DE-Juel1)164828 |b 4 |
| 700 | 1 | _ | |a Asplund, Christopher L. |0 P:(DE-HGF)0 |b 5 |
| 700 | 1 | _ | |a Eickhoff, Simon B |0 P:(DE-Juel1)131678 |b 6 |
| 700 | 1 | _ | |a Bzdok, Danilo |0 P:(DE-HGF)0 |b 7 |
| 700 | 1 | _ | |a Holmes, Avram J |0 P:(DE-HGF)0 |b 8 |
| 700 | 1 | _ | |a Yeo, B. T. Thomas |0 P:(DE-HGF)0 |b 9 |e Corresponding author |
| 773 | _ | _ | |a 10.1016/j.neuroimage.2023.120115 |g p. 120115 - |0 PERI:(DE-600)1471418-8 |p 120115 - |t NeuroImage |v 274 |y 2023 |x 1053-8119 |
| 856 | 4 | _ | |u https://juser.fz-juelich.de/record/1006999/files/1-s2.0-S1053811923002616-main.pdf |y OpenAccess |
| 909 | C | O | |o oai:juser.fz-juelich.de:1006999 |p openaire |p open_access |p VDB |p driver |p dnbdelivery |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 4 |6 P:(DE-Juel1)164828 |
| 910 | 1 | _ | |a HHU Düsseldorf |0 I:(DE-HGF)0 |b 4 |6 P:(DE-Juel1)164828 |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 6 |6 P:(DE-Juel1)131678 |
| 910 | 1 | _ | |a HHU Düsseldorf |0 I:(DE-HGF)0 |b 6 |6 P:(DE-Juel1)131678 |
| 910 | 1 | _ | |a Centre for Sleep and Cognition, Yong Loo Lin School of Medicine, National University of Singapore |0 I:(DE-HGF)0 |b 9 |6 P:(DE-HGF)0 |
| 913 | 1 | _ | |a DE-HGF |b Key Technologies |l Natural, Artificial and Cognitive Information Processing |1 G:(DE-HGF)POF4-520 |0 G:(DE-HGF)POF4-525 |3 G:(DE-HGF)POF4 |2 G:(DE-HGF)POF4-500 |4 G:(DE-HGF)POF |v Decoding Brain Organization and Dysfunction |9 G:(DE-HGF)POF4-5251 |x 0 |
| 914 | 1 | _ | |y 2023 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0160 |2 StatID |b Essential Science Indicators |d 2022-11-12 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)1190 |2 StatID |b Biological Abstracts |d 2022-11-12 |
| 915 | _ | _ | |a Creative Commons Attribution-NonCommercial-NoDerivs CC BY-NC-ND 4.0 |0 LIC:(DE-HGF)CCBYNCND4 |2 HGFVOC |
| 915 | _ | _ | |a WoS |0 StatID:(DE-HGF)0113 |2 StatID |b Science Citation Index Expanded |d 2022-11-12 |
| 915 | _ | _ | |a Fees |0 StatID:(DE-HGF)0700 |2 StatID |d 2022-11-12 |
| 915 | _ | _ | |a OpenAccess |0 StatID:(DE-HGF)0510 |2 StatID |
| 915 | _ | _ | |a Article Processing Charges |0 StatID:(DE-HGF)0561 |2 StatID |d 2022-11-12 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0501 |2 StatID |b DOAJ Seal |d 2023-05-02T08:47:40Z |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0500 |2 StatID |b DOAJ |d 2023-05-02T08:47:40Z |
| 915 | _ | _ | |a Peer Review |0 StatID:(DE-HGF)0030 |2 StatID |b DOAJ : Anonymous peer review |d 2023-05-02T08:47:40Z |
| 915 | _ | _ | |a Nationallizenz |0 StatID:(DE-HGF)0420 |2 StatID |d 2023-10-21 |w ger |
| 915 | _ | _ | |a JCR |0 StatID:(DE-HGF)0100 |2 StatID |b NEUROIMAGE : 2022 |d 2023-10-21 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0200 |2 StatID |b SCOPUS |d 2023-10-21 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0300 |2 StatID |b Medline |d 2023-10-21 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0600 |2 StatID |b Ebsco Academic Search |d 2023-10-21 |
| 915 | _ | _ | |a Peer Review |0 StatID:(DE-HGF)0030 |2 StatID |b ASC |d 2023-10-21 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0199 |2 StatID |b Clarivate Analytics Master Journal List |d 2023-10-21 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)1050 |2 StatID |b BIOSIS Previews |d 2023-10-21 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0150 |2 StatID |b Web of Science Core Collection |d 2023-10-21 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)1030 |2 StatID |b Current Contents - Life Sciences |d 2023-10-21 |
| 915 | _ | _ | |a IF >= 5 |0 StatID:(DE-HGF)9905 |2 StatID |b NEUROIMAGE : 2022 |d 2023-10-21 |
| 920 | _ | _ | |l yes |
| 920 | 1 | _ | |0 I:(DE-Juel1)INM-7-20090406 |k INM-7 |l Gehirn & Verhalten |x 0 |
| 980 | _ | _ | |a journal |
| 980 | _ | _ | |a VDB |
| 980 | _ | _ | |a UNRESTRICTED |
| 980 | _ | _ | |a I:(DE-Juel1)INM-7-20090406 |
| 980 | 1 | _ | |a FullTexts |
| Library | Collection | CLSMajor | CLSMinor | Language | Author |
|---|