TY  - CONF
AU  - Keup, Christian
AU  - Kühn, Tobias
AU  - Dahmen, David
AU  - Helias, Moritz
TI  - Transient chaotic SNR amplification
M1  - FZJ-2021-01268
PY  - 2021
N1  - The short summary video can be found online at https://vimeo.com/514398744
AB  - Strongly chaotic non-linear networks strongly separate inputs, but are believed to be useless for classification tasksbecause also irrelevant (noise) differences within any class are exacerbated, leading to bad generalization. We show thisis actually not the case during the initial time period following input presentation: During this time, the representationis dominated by expansion, but not by mixing, and larger differences (between classes) expand faster than smallerdifferences (within classes). Therefore, the representation is disentangled by the dynamics, and when classifying thenetwork state by linear readouts, the signal-to-noise ratio (SNR) actually increases, before it eventually deteriorateswhen mixing begins to dominate. We show that this is a general effect in high-dimensional non-linear chaotic systems,and demonstrate it in spiking, continuous rate, and LSTM networks. The transient SNR amplification is always fast(within 50 ms) for spiking networks, while its timescale in continuous valued networks depends on the distance to theedge of chaos. Moreover, this fast, noise-resilient transient disentanglement of representations is in line with empiricalevidence: the olfactory bulb, for example, rapidly enhances the separability of sensory representations in a singlerecurrent layer, being the initial processing stage of a relatively flat hierarchy.
T2  - COSYNE 2021
CY  - 24 Feb 2021 - 26 Feb 2021, virtual (NA)
Y2  - 24 Feb 2021 - 26 Feb 2021
M2  - virtual, NA
LB  - PUB:(DE-HGF)24
UR  - https://juser.fz-juelich.de/record/890946
ER  -