Hauptseite > Publikationsdatenbank > Empirical Comparison Between Cross-Validation and Mutation-Validation in Model Selection > print

001     1025669
005     20241023094532.0
020 _ _ |a 978-3-031-58555-5 (print)
020 _ _ |a 978-3-031-58553-1 (electronic)
024 7 _ |a 10.1007/978-3-031-58553-1_5
|2 doi
024 7 _ |a 0302-9743
|2 ISSN
024 7 _ |a 1611-3349
|2 ISSN
024 7 _ |a 10.34734/FZJ-2024-03058
|2 datacite_doi
024 7 _ |a WOS:001295920900005
|2 WOS
037 _ _ |a FZJ-2024-03058
100 1 _ |a Miliou, Ioanna
|0 P:(DE-HGF)0
|b 0
|e Editor
111 2 _ |a 22nd International Symposium on Intelligent Data Analysis, IDA 2024, Proceedings, Part II
|c Stockholm
|d 2024-04-24 - 2024-04-26
|w Sweden
245 _ _ |a Empirical Comparison Between Cross-Validation and Mutation-Validation in Model Selection
260 _ _ |a Cham
|c 2024
|b Springer Nature Switzerland
295 1 0 |a Advances in Intelligent Data Analysis XXII / Miliou, Ioanna (Editor) [https://orcid.org/0000-0002-1357-1967] ; Cham : Springer Nature Switzerland, 2024, Chapter 5 ; ISSN: 0302-9743=1611-3349 ; ISBN: 978-3-031-58555-5=978-3-031-58553-1 ; doi:10.1007/978-3-031-58553-1
300 _ _ |a 56 - 67
336 7 _ |a CONFERENCE_PAPER
|2 ORCID
336 7 _ |a Conference Paper
|0 33
|2 EndNote
336 7 _ |a INPROCEEDINGS
|2 BibTeX
336 7 _ |a conferenceObject
|2 DRIVER
336 7 _ |a Output Types/Conference Paper
|2 DataCite
336 7 _ |a Contribution to a conference proceedings
|b contrib
|m contrib
|0 PUB:(DE-HGF)8
|s 1714639942_20631
|2 PUB:(DE-HGF)
336 7 _ |a Contribution to a book
|0 PUB:(DE-HGF)7
|2 PUB:(DE-HGF)
|m contb
490 0 _ |a Lecture Notes in Computer Science
|v 14642
520 _ _ |a Mutation validation (MV) is a recently proposed approach for model selection, garnering significant interest due to its unique characteristics and potential benefits compared to the widely used cross-validation (CV) method. In this study, we empirically compared MV and k-fold CV using benchmark and real-world datasets. By employing Bayesian tests, we compared generalization estimates yielding three posterior probabilities: practical equivalence, CV superiority, and MV superiority. We also evaluated the differences in the capacity of the selected models and computational efficiency. We found that both MV and CV select models with practically equivalent generalization performance across various machine learning algorithms and the majority of benchmark datasets. MV exhibited advantages in terms of selecting simpler models and lower computational costs. However, in some cases MV selected overly simplistic models leading to underfitting and showed instability in hyperparameter selection. These limitations of MV became more evident in the evaluation of a real-world neuroscientific task of predicting sex at birth using brain functional connectivity.
536 _ _ |a 5253 - Neuroimaging (POF4-525)
|0 G:(DE-HGF)POF4-5253
|c POF4-525
|f POF IV
|x 0
536 _ _ |a 5254 - Neuroscientific Data Analytics and AI (POF4-525)
|0 G:(DE-HGF)POF4-5254
|c POF4-525
|f POF IV
|x 1
588 _ _ |a Dataset connected to CrossRef Book Series, Journals: juser.fz-juelich.de
700 1 _ |a Piatkowski, Nico
|0 P:(DE-HGF)0
|b 1
|e Editor
700 1 _ |a Papapetrou, Panagiotis
|0 P:(DE-HGF)0
|b 2
|e Editor
700 1 _ |a Yu, Jinyang
|0 P:(DE-HGF)0
|b 3
700 1 _ |a Hamdan, Sami
|0 P:(DE-Juel1)184874
|b 4
700 1 _ |a Sasse, Leonard
|0 P:(DE-Juel1)190306
|b 5
700 1 _ |a Morrison, Abigail
|0 P:(DE-Juel1)151166
|b 6
700 1 _ |a Patil, Kaustubh R.
|0 P:(DE-Juel1)172843
|b 7
|e Corresponding author
773 _ _ |a 10.1007/978-3-031-58553-1_5
856 4 _ |y OpenAccess
|u https://juser.fz-juelich.de/record/1025669/files/2311.14079v2.pdf
856 4 _ |y OpenAccess
|x icon
|u https://juser.fz-juelich.de/record/1025669/files/2311.14079v2.gif?subformat=icon
856 4 _ |y OpenAccess
|x icon-1440
|u https://juser.fz-juelich.de/record/1025669/files/2311.14079v2.jpg?subformat=icon-1440
856 4 _ |y OpenAccess
|x icon-180
|u https://juser.fz-juelich.de/record/1025669/files/2311.14079v2.jpg?subformat=icon-180
856 4 _ |y OpenAccess
|x icon-640
|u https://juser.fz-juelich.de/record/1025669/files/2311.14079v2.jpg?subformat=icon-640
909 C O |o oai:juser.fz-juelich.de:1025669
|p openaire
|p open_access
|p VDB
|p driver
|p dnbdelivery
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 4
|6 P:(DE-Juel1)184874
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 5
|6 P:(DE-Juel1)190306
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 6
|6 P:(DE-Juel1)151166
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 7
|6 P:(DE-Juel1)172843
910 1 _ |a HHU Düsseldorf
|0 I:(DE-HGF)0
|b 7
|6 P:(DE-Juel1)172843
913 1 _ |a DE-HGF
|b Key Technologies
|l Natural, Artificial and Cognitive Information Processing
|1 G:(DE-HGF)POF4-520
|0 G:(DE-HGF)POF4-525
|3 G:(DE-HGF)POF4
|2 G:(DE-HGF)POF4-500
|4 G:(DE-HGF)POF
|v Decoding Brain Organization and Dysfunction
|9 G:(DE-HGF)POF4-5253
|x 0
913 1 _ |a DE-HGF
|b Key Technologies
|l Natural, Artificial and Cognitive Information Processing
|1 G:(DE-HGF)POF4-520
|0 G:(DE-HGF)POF4-525
|3 G:(DE-HGF)POF4
|2 G:(DE-HGF)POF4-500
|4 G:(DE-HGF)POF
|v Decoding Brain Organization and Dysfunction
|9 G:(DE-HGF)POF4-5254
|x 1
914 1 _ |y 2024
915 _ _ |a DBCoverage
|0 StatID:(DE-HGF)0200
|2 StatID
|b SCOPUS
|d 2023-09-03
915 _ _ |a OpenAccess
|0 StatID:(DE-HGF)0510
|2 StatID
915 _ _ |a Nationallizenz
|0 StatID:(DE-HGF)0420
|2 StatID
|d 2023-09-03
|w ger
920 1 _ |0 I:(DE-Juel1)INM-7-20090406
|k INM-7
|l Gehirn & Verhalten
|x 0
920 1 _ |0 I:(DE-Juel1)IAS-6-20130828
|k IAS-6
|l Computational and Systems Neuroscience
|x 1
980 _ _ |a contrib
980 _ _ |a VDB
980 _ _ |a UNRESTRICTED
980 _ _ |a contb
980 _ _ |a I:(DE-Juel1)INM-7-20090406
980 _ _ |a I:(DE-Juel1)IAS-6-20130828
980 1 _ |a FullTexts

LibraryCollectionCLSMajorCLSMinorLanguageAuthor
Marc 21