001033584 001__ 1033584
001033584 005__ 20241212210724.0
001033584 037__ $$aFZJ-2024-06464
001033584 1001_ $$0P:(DE-Juel1)177675$$aUpschulte, Eric$$b0$$eCorresponding author$$ufzj
001033584 1112_ $$aINM Retreat 2024$$cJülich$$d2024-11-19 - 2024-11-20$$wGermany
001033584 245__ $$aTowards Universal Instance Segmentation Models in Biomedical Imaging
001033584 260__ $$c2024
001033584 3367_ $$033$$2EndNote$$aConference Paper
001033584 3367_ $$2BibTeX$$aINPROCEEDINGS
001033584 3367_ $$2DRIVER$$aconferenceObject
001033584 3367_ $$2ORCID$$aCONFERENCE_POSTER
001033584 3367_ $$2DataCite$$aOutput Types/Conference Poster
001033584 3367_ $$0PUB:(DE-HGF)24$$2PUB:(DE-HGF)$$aPoster$$bposter$$mposter$$s1734001959_5470$$xAfter Call
001033584 520__ $$aPrecise instance segmentation is crucial in many biomedical research fields. One key challenge is applying models to new data domains, typically involving pre-training on a larger corpus of data and fine-tuning with new annotations for each specific domain. This process is labor-intensive and requires creating and maintaining multiple branched versions of the model. Working towards universal instance segmentation models in biomedical imaging, we propose to unify domain-adapted model branches into a single multi- expert model, following a foundation model paradigm. Our goal is to replace most existing fine-tuning scenarios with prompt-based user instructions, allowing the user to clearly state the task and object classes of interest. We hypothesize that such a combined approach improves generalization, as the base model can benefit from datasets that were previously only used for fine-tuning. A key challenge in the creation of such models is to resolve training conflicts and ambiguity in a pragmatic fashion when combining different segmentation tasks, datasets, and data domains. Such conflicts can occur if datasets focus on different classes in the same domain. For example, some datasets annotate all cells, while others focus on specific cell types. A naïve combination of such sets would create an ill-posed learning problem for most models, requiring them to infer their task from their input, which is undesirable in a universal setting. Models like SAM and MedSAM highlight the potential of prompting, but often require external detectors and fine-tuning. Here, we propose to leverage prompt-based task descriptions as a tool to manipulate general model behavior, such that user instructions yield domain expert models. We test our approach by training a Contour Proposal Network (CPN) on a multi-modal data collection, including the TissueNet dataset. Prompts, such as “cell segmentation” or simply “nuclei”, modify the CPN to focus on segmenting the respective object classes, achieving a mean F1 score in TissueNet of 0.90 (0.88 for cells, 0.92 for nuclei), which is on par with specialized models and surpasses the naïve combination showing 0.84 (0.81, 0.87) without prompting. Overall, the proposed approach introduces an interactive linguistic component that enables the conflict-free composition of various segmentation datasets, thus allowing to unify previously separated segmentation tasks. With that, we consider it an important step towards universal models.
001033584 536__ $$0G:(DE-HGF)POF4-5251$$a5251 - Multilevel Brain Organization and Variability (POF4-525)$$cPOF4-525$$fPOF IV$$x0
001033584 536__ $$0G:(DE-HGF)POF4-5254$$a5254 - Neuroscientific Data Analytics and AI (POF4-525)$$cPOF4-525$$fPOF IV$$x1
001033584 536__ $$0G:(DE-HGF)InterLabs-0015$$aHIBALL - Helmholtz International BigBrain Analytics and Learning Laboratory (HIBALL) (InterLabs-0015)$$cInterLabs-0015$$x2
001033584 536__ $$0G:(DE-Juel-1)E.40401.62$$aHelmholtz AI - Helmholtz Artificial Intelligence  Coordination Unit – Local Unit FZJ (E.40401.62)$$cE.40401.62$$x3
001033584 536__ $$0G:(EU-Grant)101147319$$aEBRAINS 2.0 - EBRAINS 2.0: A Research Infrastructure to Advance Neuroscience and Brain Health (101147319)$$c101147319$$fHORIZON-INFRA-2022-SERV-B-01$$x4
001033584 7001_ $$0P:(DE-HGF)0$$aHarmeling, Stefan$$b1
001033584 7001_ $$0P:(DE-Juel1)131631$$aAmunts, Katrin$$b2$$ufzj
001033584 7001_ $$0P:(DE-Juel1)165746$$aDickscheid, Timo$$b3$$ufzj
001033584 909CO $$ooai:juser.fz-juelich.de:1033584$$popenaire$$pVDB$$pec_fundedresources
001033584 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)177675$$aForschungszentrum Jülich$$b0$$kFZJ
001033584 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)131631$$aForschungszentrum Jülich$$b2$$kFZJ
001033584 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)165746$$aForschungszentrum Jülich$$b3$$kFZJ
001033584 9131_ $$0G:(DE-HGF)POF4-525$$1G:(DE-HGF)POF4-520$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5251$$aDE-HGF$$bKey Technologies$$lNatural, Artificial and Cognitive Information Processing$$vDecoding Brain Organization and Dysfunction$$x0
001033584 9131_ $$0G:(DE-HGF)POF4-525$$1G:(DE-HGF)POF4-520$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5254$$aDE-HGF$$bKey Technologies$$lNatural, Artificial and Cognitive Information Processing$$vDecoding Brain Organization and Dysfunction$$x1
001033584 9141_ $$y2024
001033584 920__ $$lyes
001033584 9201_ $$0I:(DE-Juel1)INM-1-20090406$$kINM-1$$lStrukturelle und funktionelle Organisation des Gehirns$$x0
001033584 980__ $$aposter
001033584 980__ $$aVDB
001033584 980__ $$aI:(DE-Juel1)INM-1-20090406
001033584 980__ $$aUNRESTRICTED