SGLang 首日支持 NVIDIA Nemotron 3 Super，面向多智能体系统推理

Contents

We are excited to announce that SGLang supports NVIDIA Nemotron 3 Super on Day 0.

Nemotron 3 Super is a leading open model in the Nemotron 3 family, built for running many collaborating agents together. Agentic systems that chain planning, reasoning, and tools produce far more tokens than single-turn chat; they also need strong reasoning on every step.

Nemotron 3 Super is a 120B-parameter hybrid MoE that activates only 12B parameters per forward pass, giving you leading accuracy for coding, tool calling, and instruction following at a fraction of the cost—plus a 1M-token context so agents keep conversation and plan state in view across long workflows.

As you can see in the chart above, Nemotron 3 Super leads on the Artificial Analysis Openness index. When compared to other open models, Nemotron is fully open with open-weights, datasets, and recipes so developers can easily customize, optimize, and deploy on their infrastructure for maximum privacy and security.

In this post we walk through installing SGLang and serving Nemotron 3 Super for inference.

About Nemotron3 Super

Highest throughput efficiency in its size category and up to 5x higher throughput compared to previous Nemotron Super model (Llama Nemotron Super 1.5); Multi-Token Prediction (MTP) : By predicting several future tokens simultaneously in a single forward pass, MTP drastically accelerates the generation of long-form text; Supports Thinking Budget for optimal accuracy with minimum reasoning token generation; Accuracy: Leading accuracy on Artificial Analysis Intelligence Index in its size category

Download model weights from Hugging Face - BF16, FP8 and NVFP4; Run with SGLang for inference; Technical report to build custom, optimized models with Nemotron techniques

Installation and Quick Start

For an easier setup with SGLang, refer to our getting started cookbook, available here or through NVIDIA Brev launchable.

Run the command below to install dependencies:

We can then serve this model. The command below is configured for a 4xH200 setup. Refer to the cookbooks for detailed instructions

Once the server is up and running, you can prompt the model using the below code snippets:

Nemotron 3 Super is ideal for multi-agent and reasoning workloads

As you can see in the chart above, the model achieves leading accuracy with higher efficiency on Artificial analysis benchmarks making it a strong choice for multi-agent systems that need both efficiency and capability.

The 1M-token context is built for long-horizon agent work: agents can keep full conversation history and plan state in context, and RAG pipelines can supply large document sets in one shot. That reduces fragmentation and goal drift in multi-step workflows.

Together, this makes Super a strong choice for orchestrating and running many agents on a single node—from code generation and debugging to research summarization, alert triage, and document analysis.

Nemotron 3 Super helps you build scalable, cost-efficient multi-agent AI with high accuracy. With open weights, datasets, and recipes, you get full transparency and the flexibility to fine-tune and deploy on your own infrastructure, from workstation to cloud.

Ready to run multi-agent AI at scale?

Download Nemotron 3 Super model weights from Hugging Face - BF16, FP8 and NVFP4; Run with SGLang for inference using the cookbook and through Brev launchable; Read the Nemotron 3 Super technical report

Acknowledgement

Thanks to everyone who contributed to bringing Nemotron 3 Super to SGLang.