Journal Article

FZJ-2021-01301

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Unsupervised Learning and Clustered Connectivity Enhance Reinforcement Learning in Spiking Neural Networks

Weidel, P. ; Duarte, R. (Corresponding author)FZJ* ; Morrison, A.FZJ*

2021
Frontiers Research Foundation Lausanne

This record in other databases:      

Please use a persistent id in citations: http://hdl.handle.net/2128/28017  doi:10.3389/fncom.2021.543872

Abstract: Reinforcement learning is a paradigm that can account for how organisms learn to adapt their behavior in complex environments with sparse rewards. To partition an environment into discrete states, implementations in spiking neuronal networks typically rely on input architectures involving place cells or receptive fields specified ad hoc by the researcher. This is problematic as a model for how an organism can learn appropriate behavioral sequences in unknown environments, as it fails to account for the unsupervised and self-organized nature of the required representations. Additionally, this approach presupposes knowledge on the part of the researcher on how the environment should be partitioned and represented and scales poorly with the size or complexity of the environment. To address these issues and gain insights into how the brain generates its own task-relevant mappings, we propose a learning architecture that combines unsupervised learning on the input projections with biologically motivated clustered connectivity within the representation layer. This combination allows input features to be mapped to clusters; thus the network self-organizes to produce clearly distinguishable activity patterns that can serve as the basis for reinforcement learning on the output projections. On the basis of the MNIST and Mountain Car tasks, we show that our proposed model performs better than either a comparable unclustered network or a clustered network with static input projections. We conclude that the combination of unsupervised learning and clustered connectivity provides a generic representational substrate suitable for further computation.

---

Contributing Institute(s):

1. Computational and Systems Neuroscience (INM-6)
2. Theoretical Neuroscience (IAS-6)
3. Jara-Institut Brain structure-function relationships (INM-10)
4. JARA - HPC (JARA-HPC)

Research Program(s):

1. 523 - Neuromorphic Computing and Network Dynamics (POF4-523) (POF4-523)
2. 89574 - Theory, modelling and simulation (POF2-89574) (POF2-89574)
3. Functional Neural Architectures (jinm60_20190501) (jinm60_20190501)
4. 5232 - Computational Principles (POF4-523) (POF4-523)

Appears in the scientific report 2021

---

Database coverage:
; ; ; ; Article Processing Charges ; BIOSIS Previews ; Biological Abstracts ; Clarivate Analytics Master Journal List ; DOAJ Seal ; Essential Science Indicators ; Fees ; IF < 5 ; JCR ; PubMed Central ; SCOPUS ; Science Citation Index Expanded ; Web of Science Core Collection

---