SGLang 为 Meta 多模态模型 Muse Glimmer 提供 Day-0 支持
SGLang Adds Day-0 Support for Muse Glimmer, a Multimodal Model Built for Local Agentic Workflows - LMSYS Org
LMSYS 宣布与 Meta Superintelligence Labs 合作，为 30B 参数多模态模型 Muse Glimmer 在 SGLang 上提供 Day-0 支持，并针对本地智能体工作流推理做了专门优化。在 NVIDIA GeForce RTX 5090 上，SGLang 以 NVFP4 加 DFlash 实现最高 1,452 tok/s 总输出吞吐和 236 tok/s 单用户解码速度，DFlash 将 batch-1 交互性提升 1.9 至 4.3 倍。模型可在 RTX 5090、RTX PRO 6000、DGX Spark 的 SM120 后端以及 Apple Silicon 的 MLX 后端部署，提供 BF16、NVFP4+MXFP8、NVFP4 及 GGUF 等多种 checkpoint 格式。
推荐理由
精选AI 评分 62
SGLang 在 Muse Glimmer 发布当天给出本地部署支持与实测吞吐，读者可据此判断消费级硬件跑多模态智能体的可行性。
主题
正文
当前显示原文，中文全文译文尚未就绪。
Contents
We're excited to partner with Meta Superintelligence Labs to bring Day-0 support for Muse Glimmer to SGLang, with dedicated optimizations tailored for high-performance inference of agentic workflows on local hardware.
Highlights
Model: Muse Glimmer is a 30B-parameter multimodal dense model with a 128k+ token context window that delivers competitive performance on end-to-end agentic workflows from local hardware; Performance: SGLang delivers up to 1,452 tok/s of total output throughput and 236 tok/s of per-user decode speed (NVFP4 with DFlash) on the NVIDIA GeForce RTX 5090, powered by dedicated optimizations for SGLang's SM120 backend; Features: SGLang provides broad feature support natively for Muse Glimmer, including DFlash speculative decoding, RadixAttention prefix caching, and breakable CUDA graphs; Hardware: Developers can deploy Muse Glimmer across a variety of consumer and workstation hardware — NVIDIA GeForce RTX 5090, RTX PRO 6000, and DGX Spark via the SM120 backend, and Apple Silicon via the MLX backend
Muse Glimmer Model Architecture
Muse Glimmer is a 30B model, consisting of a 27.9B dense text decoder, a 1.9B ViT, and a GELU-based multimodal projector. The text decoder has 52 transformer layers. Each layer contains grouped-query attention with 32 query heads and two key-value heads, followed by a SwiGLU feed-forward network.
Sliding Window Attention
The decoder uses a hybrid attention pattern that interleaves three 2,048-token sliding-window layers with a full-sequence layer every fourth step for efficiency. Combining RoPE on the local windows with NoPE on the full-attention layers allows the model to extend its context length beyond its training limits.
Feature Support
Muse Glimmer can be deployed with SGLang across a wide range of hardware commonly used by developers building and running AI agents locally, including Apple Silicon devices (Mac mini and M-series MacBook Pro), NVIDIA GeForce RTX 5090 GPUs, and the NVIDIA DGX Spark. On NVIDIA SM120 platforms, SGLang leverages its optimized GEMM and FlashInfer backends for high-throughput inference, while Apple Silicon devices use the native MLX backend to deliver high-performance local serving.
Speculative Decoding with DFlash
To achieve low-latency inference, use SGLang's implementation of DFlash:
SGLang Native Optimizations
Muse Glimmer is compatible with SGLang's many native optimizations, including the low-overhead scheduler, the RadixAttention prefix cache, and breakable CUDA graphs. We have also brought several of these optimizations, including prefix caching, to the SGLang MLX backend, enabling competitive performance on Apple Silicon for agentic workloads.
We provide Muse Glimmer checkpoints in several formats to meet the needs of users with different hardware, fidelity, and system performance requirements.
Developers requiring maximum model fidelity can run the native BF16 checkpoint on a single H100 GPU. The SM120 path includes a ~19.5 GB mixed NVFP4+MXFP8 quantization recipe. An 18 GB NVFP4 checkpoint paired with a 5 GB BF16 DFlash speculator fits comfortably on a single RTX 5090, enabling high-performance deployment on NVIDIA GeForce RTX 5090 accelerators and DGX Spark.
We also provide two GGUF checkpoints. The smaller of the two is in Q4KM format with a group size of 128. This checkpoint allows for faster inference speeds and deployment on hardware with tighter memory constraints. Q4K-Dynamic delivers better model quality. For developers working on Apple Silicon devices, we provide the previously mentioned GGUF checkpoints in MLX format.
Performance Results
We measured Muse Glimmer with SGLang across seven platforms, sweeping batch sizes 1 through 8. Reported below are batch-1 interactivity (tok/s/user) and batch-8 aggregate output throughput (tok/s).
DFlash raises batch-1 interactivity by 1.9–4.3x depending on platform and precision: 3.0x or better on every configuration except the GGUF path on the RTX 5090, which gains 1.9x. The batch-8 gain is smaller and more variable, ranging from 1.4x to 4.2x. Speculative decoding is not available on the MLX backend.
Acknowledgements
We thank the teams at Meta Superintelligence Labs and the SGLang community for their collaboration in bringing Day-0 support for Muse Glimmer to SGLang.