001038638 001__ 1038638
001038638 005__ 20250203215421.0
001038638 037__ $$aFZJ-2025-01609
001038638 041__ $$aEnglish
001038638 1001_ $$0P:(DE-Juel1)140202$$aStrube, Alexandre$$b0$$eCorresponding author
001038638 1112_ $$aHelmholtz AI Conference$$cDüsseldorf$$d2025-06-12 - 2025-06-14$$gHAICON 2024$$wGermany
001038638 245__ $$aHelmholtz Blablador: An Inference Server for Scientific Large Language Models
001038638 260__ $$c2024
001038638 3367_ $$033$$2EndNote$$aConference Paper
001038638 3367_ $$2DataCite$$aOther
001038638 3367_ $$2BibTeX$$aINPROCEEDINGS
001038638 3367_ $$2DRIVER$$aconferenceObject
001038638 3367_ $$2ORCID$$aLECTURE_SPEECH
001038638 3367_ $$0PUB:(DE-HGF)6$$2PUB:(DE-HGF)$$aConference Presentation$$bconf$$mconf$$s1738566608_21854$$xOther
001038638 520__ $$aRecent advances in large language models (LLMs) like chatGPT have demonstrated their potential for generating human-like text and reasoning about topics with natural language. However, applying these advanced LLMs requires significant compute resources and expertise that are out of reach for most academic researchers. To make scientific LLMs more accessible, we have developed Helmholtz Blablador, an open-source inference server optimized for serving predictions from customized scientific LLMs.Blablador provides the serving infrastructure to make models accessible via a simple API without managing servers, firewalls, authentication or infrastructure. Researchers can add their pretrained LLMs to the central hub. Other scientists can then query the collective model catalog via web or using the popular OpenAI api to add LLM functionality in other tools, like programming IDEs.This enables a collaborative ecosystem for scientific LLMs:Researchers train models using datasets and GPUs from their own lab. No need to set up production servers. They can even provide their models with inference happening on cpus, with the use of tools like llama.cpp.Models are contributed to the Blablador hub through a web UI or API call. Blablador handles loading models and publishing models for general use.Added models become available for querying by other researchers.A model catalog displays available LLMs from different labs and research areas.Besides that, one can train, quantize, fine-tune and evaluate LLMs directly with Blablador.The inference server is available at http://helmholtz-blablador.fz-juelich.de
001038638 536__ $$0G:(DE-HGF)POF4-5112$$a5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511)$$cPOF4-511$$fPOF IV$$x0
001038638 536__ $$0G:(DE-Juel-1)E54.303.11$$aHelmholtz AI Consultant Team FB Information (E54.303.11)$$cE54.303.11$$x1
001038638 8564_ $$uhttps://haicon24.de
001038638 909CO $$ooai:juser.fz-juelich.de:1038638$$pVDB
001038638 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)140202$$aForschungszentrum Jülich$$b0$$kFZJ
001038638 9131_ $$0G:(DE-HGF)POF4-511$$1G:(DE-HGF)POF4-510$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5112$$aDE-HGF$$bKey Technologies$$lEngineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action$$vEnabling Computational- & Data-Intensive Science and Engineering$$x0
001038638 9141_ $$y2024
001038638 920__ $$lyes
001038638 9201_ $$0I:(DE-Juel1)JSC-20090406$$kJSC$$lJülich Supercomputing Center$$x0
001038638 980__ $$aconf
001038638 980__ $$aVDB
001038638 980__ $$aI:(DE-Juel1)JSC-20090406
001038638 980__ $$aUNRESTRICTED