Journal Article

FZJ-2023-01561

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png DISCONA: Distributed Sample Compression for Nearest Neighbor Algorithm

Rybicki, J. (Corresponding author)FZJ* ; Frenklach, T. ; Puzis, R.

2023
Springer Science + Business Media B.V Dordrecht [u.a.]

This record in other databases:    

Please use a persistent id in citations: http://hdl.handle.net/2128/34242  doi:10.1007/s10489-023-04482-y

Abstract: Sample compression using epsilon nets effectively reduces the number of labeled instances required for accurate classification with nearest neighbor algorithms. However, one-shot construction of an epsilon nets can be extremely challenging in large-scale distributed data sets. We explore two approaches for distributed sample compression: one where local epsilon net is constructed for each data partition and then merged during an aggregation phase, and one where a single backbone of an epsilon net is constructed from one partition and aggregates target label distributions from other partitions. Both approaches are applied to the problem of malware detection in a complex, real-world data set of Android apps using the nearest neighbor algorithm. Examination of the compression rate, computational efficiency, and predictive power shows that a single backbone of an epsilon net attains favorable performance while achieving a compression rate of 99%.

---

Contributing Institute(s):

1. Jülich Supercomputing Center (JSC)

Research Program(s):

1. 5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511) (POF4-511)

Appears in the scientific report 2023

---

Database coverage:
; ; ; Clarivate Analytics Master Journal List ; Current Contents - Engineering, Computing and Technology ; DEAL Springer ; Essential Science Indicators ; IF >= 5 ; JCR ; Nationallizenz ; SCOPUS ; Science Citation Index Expanded ; Web of Science Core Collection

---