001038039 001__ 1038039
001038039 005__ 20250203103305.0
001038039 0247_ $$2doi$$a10.48550/ARXIV.2403.10173
001038039 0247_ $$2datacite_doi$$a10.34734/FZJ-2025-01088
001038039 037__ $$aFZJ-2025-01088
001038039 1001_ $$0P:(DE-Juel1)194363$$aAhmed, Soikat Hasan$$b0$$eCorresponding author$$ufzj
001038039 245__ $$aA Hybrid SNN-ANN Network for Event-based Object Detection with Spatial and Temporal Attention
001038039 260__ $$barXiv$$c2024
001038039 3367_ $$0PUB:(DE-HGF)25$$2PUB:(DE-HGF)$$aPreprint$$bpreprint$$mpreprint$$s1738243926_31341
001038039 3367_ $$2ORCID$$aWORKING_PAPER
001038039 3367_ $$028$$2EndNote$$aElectronic Article
001038039 3367_ $$2DRIVER$$apreprint
001038039 3367_ $$2BibTeX$$aARTICLE
001038039 3367_ $$2DataCite$$aOutput Types/Working Paper
001038039 520__ $$aEvent cameras offer high temporal resolution and dynamic range with minimal motion blur, making them promising for object detection tasks. While Spiking Neural Networks (SNNs) are a natural match for event-based sensory data and enable ultra-energy efficient and low latency inference on neuromorphic hardware, Artificial Neural Networks (ANNs) tend to display more stable training dynamics and faster convergence resulting in greater task performance. Hybrid SNN-ANN approaches are a promising alternative, enabling to leverage the strengths of both SNN and ANN architectures. In this work, we introduce the first Hybrid Attention-based SNN-ANN backbone for object detection using event cameras. We propose a novel Attention-based SNN-ANN bridge module to capture sparse spatial and temporal relations from the SNN layer and convert them into dense feature maps for the ANN part of the backbone. Experimental results demonstrate that our proposed method surpasses baseline hybrid and SNN-based approaches by significant margins, with results comparable to existing ANN-based methods. Extensive ablation studies confirm the effectiveness of our proposed modules and architectural choices. These results pave the way toward a hybrid SNN-ANN architecture that achieves ANN like performance at a drastically reduced parameter budget. We implemented the SNN blocks on digital neuromorphic hardware to investigate latency and power consumption and demonstrate the feasibility of our approach.
001038039 536__ $$0G:(DE-HGF)POF4-5234$$a5234 - Emerging NC Architectures (POF4-523)$$cPOF4-523$$fPOF IV$$x0
001038039 536__ $$0G:(EU-Grant)953775$$aGREENEDGE - Taming the environmental impact of mobile networks through GREEN EDGE computing platforms (953775)$$c953775$$fH2020-MSCA-ITN-2020$$x1
001038039 536__ $$0G:(BMBF)03ZU1106CA$$aBMBF 03ZU1106CA - NeuroSys: Algorithm-Hardware Co-Design (Projekt C) - A (03ZU1106CA)$$c03ZU1106CA$$x2
001038039 536__ $$0G:(DE-Juel1)BMBF-03ZU1106CB$$aBMBF 03ZU1106CB - NeuroSys: Algorithm-Hardware Co-Design (Projekt C) - B (BMBF-03ZU1106CB)$$cBMBF-03ZU1106CB$$x3
001038039 588__ $$aDataset connected to DataCite
001038039 650_7 $$2Other$$aComputer Vision and Pattern Recognition (cs.CV)
001038039 650_7 $$2Other$$aArtificial Intelligence (cs.AI)
001038039 650_7 $$2Other$$aFOS: Computer and information sciences
001038039 7001_ $$0P:(DE-Juel1)190112$$aFinkbeiner, Jan$$b1$$ufzj
001038039 7001_ $$0P:(DE-Juel1)188273$$aNeftci, Emre$$b2$$ufzj
001038039 773__ $$a10.48550/ARXIV.2403.10173
001038039 8564_ $$uhttps://doi.org/10.48550/arXiv.2403.10173
001038039 8564_ $$uhttps://juser.fz-juelich.de/record/1038039/files/A%20Hybrid%20SNN-ANN%20Network%20for%20Event-based%20Object%20Detection%20with%20Spatial%20and%20Temporal%20Attention.pdf$$yOpenAccess
001038039 909CO $$ooai:juser.fz-juelich.de:1038039$$pdnbdelivery$$pec_fundedresources$$pVDB$$pdriver$$popen_access$$popenaire
001038039 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)194363$$aForschungszentrum Jülich$$b0$$kFZJ
001038039 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)190112$$aForschungszentrum Jülich$$b1$$kFZJ
001038039 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)188273$$aForschungszentrum Jülich$$b2$$kFZJ
001038039 9131_ $$0G:(DE-HGF)POF4-523$$1G:(DE-HGF)POF4-520$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5234$$aDE-HGF$$bKey Technologies$$lNatural, Artificial and Cognitive Information Processing$$vNeuromorphic Computing and Network Dynamics$$x0
001038039 9141_ $$y2024
001038039 915__ $$0StatID:(DE-HGF)0510$$2StatID$$aOpenAccess
001038039 9201_ $$0I:(DE-Juel1)PGI-15-20210701$$kPGI-15$$lNeuromorphic Software Eco System$$x0
001038039 980__ $$apreprint
001038039 980__ $$aVDB
001038039 980__ $$aUNRESTRICTED
001038039 980__ $$aI:(DE-Juel1)PGI-15-20210701
001038039 9801_ $$aFullTexts