TOOLORA / Local reading edition
SGLang
Inference framework for language and multimodal models, with prefix caching and multi-device serving.
Overview
SGLang is an open-source inference framework hosted by LMSYS for language, vision-language and other models. Install a runtime, load weights and launch a server, then send generation, streaming or tool-calling requests from applications. RadixAttention and prefix caching reuse shared context, while multi-GPU parallelism scales serving. The repository also contains SGLang Diffusion for image and video generation. Check current model, hardware and API compatibility lists.
Official sources and references
- Official websitehttps://docs.sglang.io/
Descriptions summarize public information; consult official websites for features, pricing and regional availability.