guest ::
| Home > Publications database > Helmholtz Blablador: An Inference Server for Scientific LargeLanguage Models |
| Home > Publications database > Helmholtz Blablador: An Inference Server for Scientific LargeLanguage Models |
| Conference Presentation (Invited) | FZJ-2025-01589 |
2024
Abstract: Recent advances in large language models (LLMs) like chatGPT have demonstrated their potentialfor generating human-like text and reasoning about topics with natural language. However, ap-plying these advanced LLMs requires significant compute resources and expertise that are out ofreach for most academic researchers. To make scientific LLMs more accessible, we have developedHelmholtz Blablador, an open-source inference server optimized for serving predictions from cus-tomized scientific LLMs.Blablador provides the serving infrastructure to make models accessible via a simple API withoutmanaging servers, firewalls, authentication or infrastructure. Researchers can add their pretrainedLLMs to the central hub. Other scientists can then query the collective model catalog via web or usingthe popular OpenAI api to add LLM functionality in other tools, like programming IDEs.This enables a collaborative ecosystem for scientific LLMs:• Researchers train models using datasets and GPUs from their own lab. No need to set up produc-tion servers. They can even provide their models with inference happening on cpus, with the useof tools like llama.cpp.• Models are contributed to the Blablador hub through a web UI or API call. Blablador handlesloading models and publishing models for general use.• Added models become available for querying by other researchers. A model catalog displaysavailable LLMs from different labs and research areas.Besides that, one can train, quantize, fine-tune and evaluate LLMs directly with Blablador.The inference server is available at http://helmholtz-blablador.fz-juelich.de
|
The record appears in these collections: |
HTML