Gast ::
| Hauptseite > Publikationsdatenbank > Transient chaotic SNR amplification |
| Hauptseite > Publikationsdatenbank > Transient chaotic SNR amplification |
| Poster (After Call) | FZJ-2021-01268 |
; ; ;
2021
Please use a persistent id in citations: http://hdl.handle.net/2128/28511
Abstract: Strongly chaotic non-linear networks strongly separate inputs, but are believed to be useless for classification tasksbecause also irrelevant (noise) differences within any class are exacerbated, leading to bad generalization. We show thisis actually not the case during the initial time period following input presentation: During this time, the representationis dominated by expansion, but not by mixing, and larger differences (between classes) expand faster than smallerdifferences (within classes). Therefore, the representation is disentangled by the dynamics, and when classifying thenetwork state by linear readouts, the signal-to-noise ratio (SNR) actually increases, before it eventually deteriorateswhen mixing begins to dominate. We show that this is a general effect in high-dimensional non-linear chaotic systems,and demonstrate it in spiking, continuous rate, and LSTM networks. The transient SNR amplification is always fast(within 50 ms) for spiking networks, while its timescale in continuous valued networks depends on the distance to theedge of chaos. Moreover, this fast, noise-resilient transient disentanglement of representations is in line with empiricalevidence: the olfactory bulb, for example, rapidly enhances the separability of sensory representations in a singlerecurrent layer, being the initial processing stage of a relatively flat hierarchy.
|
The record appears in these collections: |
PDF
Fulltext by OpenAccess repository