Gast ::
| Hauptseite > Publikationsdatenbank > Empirical Comparison between Cross-Validation and Mutation-Validation in Model Selection > print |
| Hauptseite > Publikationsdatenbank > Empirical Comparison between Cross-Validation and Mutation-Validation in Model Selection > print |
| 001 | 1021988 | ||
| 005 | 20240226075418.0 | ||
| 024 | 7 | _ | |a 10.48550/ARXIV.2311.14079 |2 doi |
| 024 | 7 | _ | |a 10.34734/FZJ-2024-01127 |2 datacite_doi |
| 037 | _ | _ | |a FZJ-2024-01127 |
| 100 | 1 | _ | |a Yu, Jinyang |0 P:(DE-Juel1)192490 |b 0 |e Corresponding author |
| 245 | _ | _ | |a Empirical Comparison between Cross-Validation and Mutation-Validation in Model Selection |
| 260 | _ | _ | |c 2023 |b arXiv |
| 336 | 7 | _ | |a Preprint |b preprint |m preprint |0 PUB:(DE-HGF)25 |s 1706624922_2473 |2 PUB:(DE-HGF) |
| 336 | 7 | _ | |a WORKING_PAPER |2 ORCID |
| 336 | 7 | _ | |a Electronic Article |0 28 |2 EndNote |
| 336 | 7 | _ | |a preprint |2 DRIVER |
| 336 | 7 | _ | |a ARTICLE |2 BibTeX |
| 336 | 7 | _ | |a Output Types/Working Paper |2 DataCite |
| 520 | _ | _ | |a Mutation validation (MV) is a recently proposed approach for model selection, garnering significant interest due to its unique characteristics and potential benefits compared to the widely used cross-validation (CV) method. In this study, we empirically compared MV and $k$-fold CV using benchmark and real-world datasets. By employing Bayesian tests, we compared generalization estimates yielding three posterior probabilities: practical equivalence, CV superiority, and MV superiority. We also evaluated the differences in the capacity of the selected models and computational efficiency. We found that both MV and CV select models with practically equivalent generalization performance across various machine learning algorithms and the majority of benchmark datasets. MV exhibited advantages in terms of selecting simpler models and lower computational costs. However, in some cases MV selected overly simplistic models leading to underfitting and showed instability in hyperparameter selection. These limitations of MV became more evident in the evaluation of a real-world neuroscientific task of predicting sex at birth using brain functional connectivity. |
| 536 | _ | _ | |a 5254 - Neuroscientific Data Analytics and AI (POF4-525) |0 G:(DE-HGF)POF4-5254 |c POF4-525 |f POF IV |x 0 |
| 588 | _ | _ | |a Dataset connected to DataCite |
| 650 | _ | 7 | |a Machine Learning (cs.LG) |2 Other |
| 650 | _ | 7 | |a Machine Learning (stat.ML) |2 Other |
| 650 | _ | 7 | |a FOS: Computer and information sciences |2 Other |
| 700 | 1 | _ | |a Hamdan, Sami |0 P:(DE-Juel1)184874 |b 1 |u fzj |
| 700 | 1 | _ | |a Sasse, Leonard |0 P:(DE-Juel1)190306 |b 2 |u fzj |
| 700 | 1 | _ | |a Morrison, Abigail |0 P:(DE-Juel1)151166 |b 3 |u fzj |
| 700 | 1 | _ | |a Patil, Kaustubh R. |0 P:(DE-Juel1)172843 |b 4 |u fzj |
| 773 | _ | _ | |a 10.48550/ARXIV.2311.14079 |
| 856 | 4 | _ | |y OpenAccess |u https://juser.fz-juelich.de/record/1021988/files/preprint.pdf |
| 856 | 4 | _ | |y OpenAccess |x icon |u https://juser.fz-juelich.de/record/1021988/files/preprint.gif?subformat=icon |
| 856 | 4 | _ | |y OpenAccess |x icon-1440 |u https://juser.fz-juelich.de/record/1021988/files/preprint.jpg?subformat=icon-1440 |
| 856 | 4 | _ | |y OpenAccess |x icon-180 |u https://juser.fz-juelich.de/record/1021988/files/preprint.jpg?subformat=icon-180 |
| 856 | 4 | _ | |y OpenAccess |x icon-640 |u https://juser.fz-juelich.de/record/1021988/files/preprint.jpg?subformat=icon-640 |
| 909 | C | O | |o oai:juser.fz-juelich.de:1021988 |p openaire |p open_access |p VDB |p driver |p dnbdelivery |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 1 |6 P:(DE-Juel1)184874 |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 2 |6 P:(DE-Juel1)190306 |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 3 |6 P:(DE-Juel1)151166 |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 4 |6 P:(DE-Juel1)172843 |
| 913 | 1 | _ | |a DE-HGF |b Key Technologies |l Natural, Artificial and Cognitive Information Processing |1 G:(DE-HGF)POF4-520 |0 G:(DE-HGF)POF4-525 |3 G:(DE-HGF)POF4 |2 G:(DE-HGF)POF4-500 |4 G:(DE-HGF)POF |v Decoding Brain Organization and Dysfunction |9 G:(DE-HGF)POF4-5254 |x 0 |
| 914 | 1 | _ | |y 2023 |
| 915 | _ | _ | |a OpenAccess |0 StatID:(DE-HGF)0510 |2 StatID |
| 920 | _ | _ | |l yes |
| 920 | 1 | _ | |0 I:(DE-Juel1)INM-7-20090406 |k INM-7 |l Gehirn & Verhalten |x 0 |
| 980 | _ | _ | |a preprint |
| 980 | _ | _ | |a VDB |
| 980 | _ | _ | |a UNRESTRICTED |
| 980 | _ | _ | |a I:(DE-Juel1)INM-7-20090406 |
| 980 | 1 | _ | |a FullTexts |
| Library | Collection | CLSMajor | CLSMinor | Language | Author |
|---|