Journal Article

FZJ-2022-05381

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Decomposing neural networks as mappings of correlation functions

Fischer, K. (Corresponding author)FZJ* ; René, A.FZJ* ; Keup, C.FZJ* ; Layer, M.FZJ* ; Dahmen, D.FZJ* ; Helias, M.FZJ*

2022
APS College Park, MD

This record in other databases:    

Please use a persistent id in citations: http://hdl.handle.net/2128/32946  doi:10.1103/PhysRevResearch.4.043143

Abstract: Understanding the functional principles of information processing in deep neural networks continues to be a challenge, in particular for networks with trained and thus nonrandom weights. To address this issue, we study the mapping between probability distributions implemented by a deep feed-forward network. We characterize this mapping as an iterated transformation of distributions, where the nonlinearity in each layer transfers information between different orders of correlation functions. This allows us to identify essential statistics in the data, as well as different information representations that can be used by neural networks. Applied to an XOR task and to MNIST, we show that correlations up to second order predominantly capture the information processing in the internal layers, while the input layer also extracts higher-order correlations from the data. This analysis provides a quantitative and explainable perspective on classification.

---

Contributing Institute(s):

1. Computational and Systems Neuroscience (INM-6)
2. Theoretical Neuroscience (IAS-6)
3. Jara-Institut Brain structure-function relationships (INM-10)

Research Program(s):

1. 5232 - Computational Principles (POF4-523) (POF4-523)
2. 5234 - Emerging NC Architectures (POF4-523) (POF4-523)
3. RenormalizedFlows - Transparent Deep Learning with Renormalized Flows (BMBF-01IS19077A) (BMBF-01IS19077A)
4. MSNN - Theory of multi-scale neuronal networks (HGF-SMHB-2014-2018) (HGF-SMHB-2014-2018)
5. ACA - Advanced Computing Architectures (SO-092) (SO-092)
6. neuroIC002 - Recurrence and stochasticity for neuro-inspired computation (EXS-SF-neuroIC002) (EXS-SF-neuroIC002)

Appears in the scientific report 2022

---

Database coverage:
; ; ; ; Clarivate Analytics Master Journal List ; DOAJ Seal ; Emerging Sources Citation Index ; SCOPUS ; Web of Science Core Collection

---