000863864 001__ 863864
000863864 005__ 20240313103130.0
000863864 0247_ $$2doi$$a10.3389/fncom.2019.00046
000863864 0247_ $$2Handle$$a2128/23048
000863864 0247_ $$2altmetric$$aaltmetric:64518255
000863864 0247_ $$2pmid$$apmid:31427939
000863864 0247_ $$2WOS$$aWOS:000478905500001
000863864 037__ $$aFZJ-2019-03835
000863864 082__ $$a610
000863864 1001_ $$0P:(DE-Juel1)178920$$aJordan, Jakob$$b0$$eCorresponding author$$ufzj
000863864 245__ $$aA closed-loop toolchain for neural network simulations of learning autonomous agents
000863864 260__ $$aLausanne$$bFrontiers Research Foundation$$c2019
000863864 3367_ $$2DRIVER$$aarticle
000863864 3367_ $$2DataCite$$aOutput Types/Journal article
000863864 3367_ $$0PUB:(DE-HGF)16$$2PUB:(DE-HGF)$$aJournal Article$$bjournal$$mjournal$$s1569917456_13644
000863864 3367_ $$2BibTeX$$aARTICLE
000863864 3367_ $$2ORCID$$aJOURNAL_ARTICLE
000863864 3367_ $$00$$2EndNote$$aJournal Article
000863864 520__ $$aNeural network simulation is an important tool for generating and evaluating hypotheses on the structure, dynamics and function of neural circuits. For scientific questions addressing organisms operating autonomously in their environments, in particular where learning is involved, it is crucial to be able to operate such simulations in a closed-loop fashion. In such a set-up, the neural agent continuously receives sensory stimuli from the environment and provides motor signals that manipulate the environment or move the agent within it. So far, most studies requiring such functionality have been conducted with custom simulation scripts and manually implemented tasks. This makes it difficult for other researchers to reproduce and build upon previous work and nearly impossible to compare the performance of different learning architectures. In this work, we present a novel approach to solve this problem, connecting benchmark tools from the field of machine learning and state-of-the-art neural network simulators from computational neuroscience. The resulting toolchain enables researchers in both fields to make use of well-tested high-performance simulation software supporting biologically plausible neuron, synapse and network models and allows them to evaluate and compare their approach on the basis of standardized environments with various levels of complexity. We demonstrate the functionality of the toolchain by implementing a neuronal actor-critic architecture for reinforcement learning in the NEST simulator and successfully training it on two different environments from the OpenAI Gym. We compare its performance to a previously suggested neural network model of reinforcement learning in the basal ganglia and a generic Q-learning algorithm.
000863864 536__ $$0G:(DE-HGF)POF3-574$$a574 - Theory, modelling and simulation (POF3-574)$$cPOF3-574$$fPOF III$$x0
000863864 536__ $$0G:(DE-Juel1)BMBF-01GQ1343$$aRL-BRD-J - Neural network mechanisms of reinforcement learning (BMBF-01GQ1343)$$cBMBF-01GQ1343$$x1
000863864 536__ $$0G:(DE-HGF)B1175.01.12$$aW2Morrison - W2/W3 Professorinnen Programm der Helmholtzgemeinschaft (B1175.01.12)$$cB1175.01.12$$x2
000863864 536__ $$0G:(DE-Juel1)HGF-SMHB-2013-2017$$aSMHB - Supercomputing and Modelling for the Human Brain (HGF-SMHB-2013-2017)$$cHGF-SMHB-2013-2017$$fSMHB$$x3
000863864 536__ $$0G:(EU-Grant)785907$$aHBP SGA2 - Human Brain Project Specific Grant Agreement 2 (785907)$$c785907$$fH2020-SGA-FETFLAG-HBP-2017$$x4
000863864 536__ $$0G:(EU-Grant)720270$$aHBP SGA1 - Human Brain Project Specific Grant Agreement 1 (720270)$$c720270$$fH2020-Adhoc-2014-20$$x5
000863864 7001_ $$0P:(DE-Juel1)162278$$aWeidel, Philipp$$b1$$ufzj
000863864 7001_ $$0P:(DE-Juel1)151166$$aMorrison, Abigail$$b2$$ufzj
000863864 773__ $$0PERI:(DE-600)2452964-3$$a10.3389/fncom.2019.00046$$p46$$tFrontiers in computational neuroscience$$v13$$x1662-5188$$y2019
000863864 8564_ $$uhttps://juser.fz-juelich.de/record/863864/files/2019-0192678-4.pdf
000863864 8564_ $$uhttps://juser.fz-juelich.de/record/863864/files/2019-0192678-4.pdf?subformat=pdfa$$xpdfa
000863864 8564_ $$uhttps://juser.fz-juelich.de/record/863864/files/fncom-13-00046.pdf$$yOpenAccess
000863864 8564_ $$uhttps://juser.fz-juelich.de/record/863864/files/fncom-13-00046.pdf?subformat=pdfa$$xpdfa$$yOpenAccess
000863864 8767_ $$82019-0192678-4$$92019-06-25$$d2019-07-16$$eAPC$$jDeposit$$lDeposit: Frontiers$$z2507.50 USD
000863864 909CO $$ooai:juser.fz-juelich.de:863864$$pdnbdelivery$$popenCost$$pec_fundedresources$$pVDB$$pdriver$$pOpenAPC$$popen_access$$popenaire
000863864 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)178920$$aForschungszentrum Jülich$$b0$$kFZJ
000863864 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)162278$$aForschungszentrum Jülich$$b1$$kFZJ
000863864 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)151166$$aForschungszentrum Jülich$$b2$$kFZJ
000863864 9131_ $$0G:(DE-HGF)POF3-574$$1G:(DE-HGF)POF3-570$$2G:(DE-HGF)POF3-500$$3G:(DE-HGF)POF3$$4G:(DE-HGF)POF$$aDE-HGF$$bKey Technologies$$lDecoding the Human Brain$$vTheory, modelling and simulation$$x0
000863864 9141_ $$y2019
000863864 915__ $$0StatID:(DE-HGF)0200$$2StatID$$aDBCoverage$$bSCOPUS
000863864 915__ $$0StatID:(DE-HGF)1050$$2StatID$$aDBCoverage$$bBIOSIS Previews
000863864 915__ $$0LIC:(DE-HGF)CCBY4$$2HGFVOC$$aCreative Commons Attribution CC BY 4.0
000863864 915__ $$0StatID:(DE-HGF)0100$$2StatID$$aJCR$$bFRONT COMPUT NEUROSC : 2017
000863864 915__ $$0StatID:(DE-HGF)0501$$2StatID$$aDBCoverage$$bDOAJ Seal
000863864 915__ $$0StatID:(DE-HGF)0500$$2StatID$$aDBCoverage$$bDOAJ
000863864 915__ $$0StatID:(DE-HGF)0111$$2StatID$$aWoS$$bScience Citation Index Expanded
000863864 915__ $$0StatID:(DE-HGF)0150$$2StatID$$aDBCoverage$$bWeb of Science Core Collection
000863864 915__ $$0StatID:(DE-HGF)9900$$2StatID$$aIF < 5
000863864 915__ $$0StatID:(DE-HGF)0510$$2StatID$$aOpenAccess
000863864 915__ $$0StatID:(DE-HGF)0030$$2StatID$$aPeer Review$$bDOAJ : Blind peer review
000863864 915__ $$0StatID:(DE-HGF)0300$$2StatID$$aDBCoverage$$bMedline
000863864 915__ $$0StatID:(DE-HGF)0320$$2StatID$$aDBCoverage$$bPubMed Central
000863864 915__ $$0StatID:(DE-HGF)0199$$2StatID$$aDBCoverage$$bClarivate Analytics Master Journal List
000863864 9201_ $$0I:(DE-Juel1)INM-6-20090406$$kINM-6$$lComputational and Systems Neuroscience$$x0
000863864 9201_ $$0I:(DE-Juel1)IAS-6-20130828$$kIAS-6$$lTheoretical Neuroscience$$x1
000863864 9201_ $$0I:(DE-Juel1)INM-10-20170113$$kINM-10$$lJara-Institut Brain structure-function relationships$$x2
000863864 9801_ $$aAPC
000863864 9801_ $$aFullTexts
000863864 980__ $$ajournal
000863864 980__ $$aVDB
000863864 980__ $$aUNRESTRICTED
000863864 980__ $$aI:(DE-Juel1)INM-6-20090406
000863864 980__ $$aI:(DE-Juel1)IAS-6-20130828
000863864 980__ $$aI:(DE-Juel1)INM-10-20170113
000863864 980__ $$aAPC
000863864 981__ $$aI:(DE-Juel1)IAS-6-20130828