Gast ::
| Hauptseite > Publikationsdatenbank > Dendritic modulation for multitask representation learning in deep feedforward networks > print |
| Hauptseite > Publikationsdatenbank > Dendritic modulation for multitask representation learning in deep feedforward networks > print |
| 001 | 1008841 | ||
| 005 | 20240313103125.0 | ||
| 024 | 7 | _ | |a 10.34734/FZJ-2023-02506 |2 datacite_doi |
| 037 | _ | _ | |a FZJ-2023-02506 |
| 100 | 1 | _ | |a Wybo, Willem |0 P:(DE-Juel1)186881 |b 0 |e Corresponding author |u fzj |
| 111 | 2 | _ | |a Cosyne 2023 |c Montreal |d 2023-03-08 - 2023-03-16 |w Canada |
| 245 | _ | _ | |a Dendritic modulation for multitask representation learning in deep feedforward networks |
| 260 | _ | _ | |c 2023 |
| 336 | 7 | _ | |a Conference Paper |0 33 |2 EndNote |
| 336 | 7 | _ | |a INPROCEEDINGS |2 BibTeX |
| 336 | 7 | _ | |a conferenceObject |2 DRIVER |
| 336 | 7 | _ | |a CONFERENCE_POSTER |2 ORCID |
| 336 | 7 | _ | |a Output Types/Conference Poster |2 DataCite |
| 336 | 7 | _ | |a Poster |b poster |m poster |0 PUB:(DE-HGF)24 |s 1689922565_8663 |2 PUB:(DE-HGF) |x After Call |
| 520 | _ | _ | |a Feedforward sensory processing in the brain is generally construed as proceeding through a hierar- chy of layers, each constructing increasingly abstract and invariant representations of sensory inputs. This interpretation is at odds with the observation that activity in sensory processing layers is heavily modulated by contextual signals, such as cross modal information or internal mental states [1]. While it is tempting to assume that such modulations bias the feedforward processing pathway towards de- tection of relevant input features given a context, this induces a dependence on the contextual state in hidden representations at any given layer. The next processing layer in the hierarchy thus has to be able to extract relevant information for each possible context. For this reason, most machine learning approaches to multitask learning apply task-specific output networks to context-independent representations of the inputs, generated by a shared trunk network.Here, we show that a network motif, where a layer of modulated hidden neurons targets an out- put neuron through task-independent feedforward weights, solves multitask learning problems, and that this network motif can be implemented with biophysically realistic neurons that receive context- modulating synaptic inputs on dendritic branches. The dendritic synapses in this motif evolve ac- cording to a Hebbian plasticity rule modulated by a global error signal. We then embed such a motif in each layer of a deep feedforward network, where it generates task-modulated representations of sensory inputs. To learn feedforward weights to the next layer in the network, we apply a contrastive learning objective that predicts whether representations originate either from different inputs, or from different task-modulations of the same input. This self-supervised approach results in deep represen- tation learning of feedforward weights that accommodate a multitude of contexts, without relying on error backpropagation between layers. |
| 536 | _ | _ | |a 5232 - Computational Principles (POF4-523) |0 G:(DE-HGF)POF4-5232 |c POF4-523 |f POF IV |x 0 |
| 536 | _ | _ | |a HBP SGA1 - Human Brain Project Specific Grant Agreement 1 (720270) |0 G:(EU-Grant)720270 |c 720270 |f H2020-Adhoc-2014-20 |x 1 |
| 536 | _ | _ | |a HBP SGA2 - Human Brain Project Specific Grant Agreement 2 (785907) |0 G:(EU-Grant)785907 |c 785907 |f H2020-SGA-FETFLAG-HBP-2017 |x 2 |
| 536 | _ | _ | |a HBP SGA3 - Human Brain Project Specific Grant Agreement 3 (945539) |0 G:(EU-Grant)945539 |c 945539 |f H2020-SGA-FETFLAG-HBP-2019 |x 3 |
| 536 | _ | _ | |a SDS005 - Towards an integrated data science of complex natural systems (PF-JARA-SDS005) |0 G:(DE-Juel-1)PF-JARA-SDS005 |c PF-JARA-SDS005 |x 4 |
| 536 | _ | _ | |a neuroIC002 - Recurrence and stochasticity for neuro-inspired computation (EXS-SF-neuroIC002) |0 G:(DE-82)EXS-SF-neuroIC002 |c EXS-SF-neuroIC002 |x 5 |
| 700 | 1 | _ | |a Tran, Viet Anh Khoa |0 P:(DE-Juel1)192408 |b 1 |u fzj |
| 700 | 1 | _ | |a Tsai, Matthias |0 P:(DE-HGF)0 |b 2 |
| 700 | 1 | _ | |a Illing, Bernd |0 P:(DE-HGF)0 |b 3 |
| 700 | 1 | _ | |a Jordan, Jakob |0 P:(DE-HGF)0 |b 4 |
| 700 | 1 | _ | |a Senn, Walter |0 P:(DE-HGF)0 |b 5 |
| 700 | 1 | _ | |a Morrison, Abigail |0 P:(DE-Juel1)151166 |b 6 |u fzj |
| 856 | 4 | _ | |u https://juser.fz-juelich.de/record/1008841/files/Cosyne_2023_dendritic_modulation_abstract%20%281%29.pdf |y OpenAccess |
| 909 | C | O | |o oai:juser.fz-juelich.de:1008841 |p openaire |p open_access |p VDB |p driver |p ec_fundedresources |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 0 |6 P:(DE-Juel1)186881 |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 1 |6 P:(DE-Juel1)192408 |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 6 |6 P:(DE-Juel1)151166 |
| 913 | 1 | _ | |a DE-HGF |b Key Technologies |l Natural, Artificial and Cognitive Information Processing |1 G:(DE-HGF)POF4-520 |0 G:(DE-HGF)POF4-523 |3 G:(DE-HGF)POF4 |2 G:(DE-HGF)POF4-500 |4 G:(DE-HGF)POF |v Neuromorphic Computing and Network Dynamics |9 G:(DE-HGF)POF4-5232 |x 0 |
| 914 | 1 | _ | |y 2023 |
| 915 | _ | _ | |a OpenAccess |0 StatID:(DE-HGF)0510 |2 StatID |
| 920 | 1 | _ | |0 I:(DE-Juel1)INM-6-20090406 |k INM-6 |l Computational and Systems Neuroscience |x 0 |
| 920 | 1 | _ | |0 I:(DE-Juel1)IAS-6-20130828 |k IAS-6 |l Theoretical Neuroscience |x 1 |
| 920 | 1 | _ | |0 I:(DE-Juel1)INM-10-20170113 |k INM-10 |l Jara-Institut Brain structure-function relationships |x 2 |
| 980 | 1 | _ | |a FullTexts |
| 980 | _ | _ | |a poster |
| 980 | _ | _ | |a VDB |
| 980 | _ | _ | |a UNRESTRICTED |
| 980 | _ | _ | |a I:(DE-Juel1)INM-6-20090406 |
| 980 | _ | _ | |a I:(DE-Juel1)IAS-6-20130828 |
| 980 | _ | _ | |a I:(DE-Juel1)INM-10-20170113 |
| 981 | _ | _ | |a I:(DE-Juel1)IAS-6-20130828 |
| Library | Collection | CLSMajor | CLSMinor | Language | Author |
|---|