001031452 001__ 1031452
001031452 005__ 20241107210038.0
001031452 037__ $$aFZJ-2024-05671
001031452 041__ $$aEnglish
001031452 1001_ $$0P:(DE-Juel1)177675$$aUpschulte, Eric$$b0$$eCorresponding author$$ufzj
001031452 1112_ $$a8th BigBrain Workshop$$cPadua$$d2024-09-09 - 2024-09-11$$wItaly
001031452 245__ $$aTowards Universal Instance Segmentation Models in Biomedical Imaging
001031452 260__ $$c2024
001031452 3367_ $$033$$2EndNote$$aConference Paper
001031452 3367_ $$2DataCite$$aOther
001031452 3367_ $$2BibTeX$$aINPROCEEDINGS
001031452 3367_ $$2DRIVER$$aconferenceObject
001031452 3367_ $$2ORCID$$aLECTURE_SPEECH
001031452 3367_ $$0PUB:(DE-HGF)6$$2PUB:(DE-HGF)$$aConference Presentation$$bconf$$mconf$$s1730976784_30533$$xAfter Call
001031452 502__ $$cHHU Düssledorf
001031452 520__ $$aPrecise instance segmentation is a critical part of many fields of research in biomedical imaging. One key challenge is applying models to new data domains, typically involving pre-training a model on a larger corpus of data and fine-tuning it with new annotations for each specific domain. This process is labor- intensive and requires creating and maintaining multiple branched versions of the model. Working towards universal instance segmentation models in biomedical imaging, we propose to unify domain-adapted model branches into a single multi-expert model, following a foundation model paradigm. Our goal is to replace most existing fine-tuning scenarios with prompt-based user instructions, allowing the user to clearly state the task and object classes of interest. We hypothesize that such a combined approach improves generalization, as the base model can benefit from datasets that were previously only used for fine-tuning. A key challenge in the creation of such models is to resolve training conflicts and ambiguity in a pragmatic fashion when combining different segmentation tasks, datasets, and data domains. Such conflicts can occur if datasets focus on different classes in the same domain. For example, some datasets annotate all cells in microscopy images, while others focus on cells of a specific cell type. A naïve combination of such sets would create an ill-posed learning problem for most models, requiring them to infer their task from their input, which is undesirable in a universal setting. Models like SAM and MedSAM highlight the potential of prompting, but often require external detectors and fine-tuning. Here, we propose to leverage prompt-based task descriptions as a tool to manipulate general model behavior, such that user instructions yield domain expert models. We test our approach by training a Contour Proposal Network (CPN) on a multi-modal data collection, including the TissueNet dataset. Prompts, such as “cell segmentation” or simply “nuclei”, modulate underlying features, allowing the CPN to segment the respective object classes in TissueNet with a mean F1 score of 0.90 (0.88 for cells, 0.92 for nuclei), compared to 0.84 (0.81, 0.87) without prompting. Overall, the proposed approach introduces an interactive linguistic component that allows the conflict-free composition of various segmentation datasets, thus allowing to unify previously separated segmentation tasks. With that, we consider it an important step towards universal models.
001031452 536__ $$0G:(DE-HGF)POF4-5254$$a5254 - Neuroscientific Data Analytics and AI (POF4-525)$$cPOF4-525$$fPOF IV$$x0
001031452 536__ $$0G:(DE-HGF)InterLabs-0015$$aHIBALL - Helmholtz International BigBrain Analytics and Learning Laboratory (HIBALL) (InterLabs-0015)$$cInterLabs-0015$$x1
001031452 536__ $$0G:(EU-Grant)101147319$$aEBRAINS 2.0 - EBRAINS 2.0: A Research Infrastructure to Advance Neuroscience and Brain Health (101147319)$$c101147319$$fHORIZON-INFRA-2022-SERV-B-01$$x2
001031452 536__ $$0G:(DE-Juel-1)E.40401.62$$aHelmholtz AI - Helmholtz Artificial Intelligence  Coordination Unit – Local Unit FZJ (E.40401.62)$$cE.40401.62$$x3
001031452 7001_ $$0P:(DE-HGF)0$$aHarmeling, Stefan$$b1
001031452 7001_ $$0P:(DE-Juel1)131631$$aAmunts, Katrin$$b2$$ufzj
001031452 7001_ $$0P:(DE-Juel1)165746$$aDickscheid, Timo$$b3$$ufzj
001031452 8564_ $$uhttps://events.hifis.net/event/1416/contributions/11283/
001031452 909CO $$ooai:juser.fz-juelich.de:1031452$$popenaire$$pVDB$$pec_fundedresources
001031452 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)177675$$aForschungszentrum Jülich$$b0$$kFZJ
001031452 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)131631$$aForschungszentrum Jülich$$b2$$kFZJ
001031452 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)165746$$aForschungszentrum Jülich$$b3$$kFZJ
001031452 9131_ $$0G:(DE-HGF)POF4-525$$1G:(DE-HGF)POF4-520$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5254$$aDE-HGF$$bKey Technologies$$lNatural, Artificial and Cognitive Information Processing$$vDecoding Brain Organization and Dysfunction$$x0
001031452 9141_ $$y2024
001031452 920__ $$lyes
001031452 9201_ $$0I:(DE-Juel1)INM-1-20090406$$kINM-1$$lStrukturelle und funktionelle Organisation des Gehirns$$x0
001031452 980__ $$aconf
001031452 980__ $$aVDB
001031452 980__ $$aI:(DE-Juel1)INM-1-20090406
001031452 980__ $$aUNRESTRICTED