TY  - JOUR
AU  - Jordan, Jakob
AU  - Weidel, Philipp
AU  - Morrison, Abigail
TI  - A closed-loop toolchain for neural network simulations of learning autonomous agents
JO  - Frontiers in computational neuroscience
VL  - 13
SN  - 1662-5188
CY  - Lausanne
PB  - Frontiers Research Foundation
M1  - FZJ-2019-03835
SP  - 46
PY  - 2019
AB  - Neural network simulation is an important tool for generating and evaluating hypotheses on the structure, dynamics and function of neural circuits. For scientific questions addressing organisms operating autonomously in their environments, in particular where learning is involved, it is crucial to be able to operate such simulations in a closed-loop fashion. In such a set-up, the neural agent continuously receives sensory stimuli from the environment and provides motor signals that manipulate the environment or move the agent within it. So far, most studies requiring such functionality have been conducted with custom simulation scripts and manually implemented tasks. This makes it difficult for other researchers to reproduce and build upon previous work and nearly impossible to compare the performance of different learning architectures. In this work, we present a novel approach to solve this problem, connecting benchmark tools from the field of machine learning and state-of-the-art neural network simulators from computational neuroscience. The resulting toolchain enables researchers in both fields to make use of well-tested high-performance simulation software supporting biologically plausible neuron, synapse and network models and allows them to evaluate and compare their approach on the basis of standardized environments with various levels of complexity. We demonstrate the functionality of the toolchain by implementing a neuronal actor-critic architecture for reinforcement learning in the NEST simulator and successfully training it on two different environments from the OpenAI Gym. We compare its performance to a previously suggested neural network model of reinforcement learning in the basal ganglia and a generic Q-learning algorithm.
LB  - PUB:(DE-HGF)16
C6  - pmid:31427939
UR  - <Go to ISI:>//WOS:000478905500001
DO  - DOI:10.3389/fncom.2019.00046
UR  - https://juser.fz-juelich.de/record/863864
ER  -