NVIDIA Nemotron Nano 2 VL 发布并获 vLLM 支持
Run Multimodal Reasoning Agents with NVIDIA Nemotron on vLLM | vLLM Blog
NVIDIA Nemotron Nano 2 VL 开源视觉语言模型发布，面向视频理解与文档智能，并获 vLLM 推理支持。模型采用混合 Transformer–Mamba 架构，通过 EVS 剪除冗余视频帧以减少 token。vLLM 夜间版本支持以 BF16、FP8 和 FP4 精度部署 OpenAI 兼容的推理服务器。
正文
当前显示原文，中文全文译文尚未就绪。
Leading multimodal model for efficient video understanding and document intelligence; Improving Efficiency with EVS; About Nemotron Nano 2 VL; Run optimized inference with vLLM; Install vLLM; Deploy and query the inference server
We are excited to release NVIDIA Nemotron Nano 2 VL, supported by vLLM. This open vision language model (VLM) is built for video understanding and document intelligence.
Nemotron Nano 2 VL uses a hybrid Transformer–Mamba design and delivers higher throughput while maintaining state-of-the-art multimodal reasoning accuracy. The model also features Efficient Video Sampling (EVS), a new technique that reduces redundant tokens generation for video workloads, allowing processing of more videos with higher efficiency.
In this blog post, we’ll explore how Nemotron Nano 2 VL advances video understanding and document intelligence, showcase real-world use cases and benchmark results, and guide you through getting started with vLLM for inference to unlock high-efficiency multimodal AI at scale.
Leading multimodal model for efficient video understanding and document intelligence
NVIDIA Nemotron Nano 2 VL brings both video understanding and document intelligence capabilities together in a single, highly efficient model. Built on the hybrid Transformer–Mamba architecture, it combines the reasoning strength of Transformer models with the compute efficiency of Mamba, achieving high throughput and low latency, allowing it to process multi-image inputs faster.
Trained on NVIDIA-curated, high-quality multimodal data, Nemotron Nano 2 VL leads in video understanding and document intelligence benchmarks such as MMMU, MathVista, AI2D, OCRBench, OCRBench-v2, OCR-Reasoning, ChartQA, DocVQA, and Video-MME, delivering top-tier accuracy in multimodal reasoning, character recognition, chart reasoning, and visual question answering. This makes it ideal for building multimodal applications that automate data extraction and comprehension across videos, documents, forms, and charts with enterprise-grade precision.
Improving Efficiency with EVS
With EVS, the model achieves higher throughput and faster response times without sacrificing accuracy. EVS technique prunes redundant frames, preserving semantic richness while enabling longer video processing efficiently. As a result, enterprises can analyze hours of footage, from meetings and training sessions to customer calls, in minutes, gaining actionable insights faster and at lower cost.
About Nemotron Nano 2 VL
Architecture:
CRADIOH-V2 based Vision Encoder; Efficient video sampling as token compression module; Hybrid Transformer-Mamba Architecture - Nemotron Nano 2 LLM backbone with reasoning; Accuracy:
Download model weights from Hugging Face - BF16, FP8, FP4-QAD; Run with vLLM for inference; Technical report to build custom, optimized models with Nemotron techniques
Run optimized inference with vLLM
This guide demonstrates how to run Nemotron Nano 2 VL on vLLM, achieving accelerated inference and serving concurrent requests efficiently with BF16, FP8 and FP4 precision support.
Install vLLM
The support for Nemotron Nano 2 VL is available in the nightly version of vLLM. Run the command below to install vLLM:
Deploy and query the inference server
Deploy an OpenAI-compatible inference server with vLLM by running the following commands for BF16, FP8 and FP4 precision:
Once the server is up and running, you can prompt the model using the below code snippet:
For more examples, check out our vLLM cookbook and vLLM recipe for Nemotron Nano 2 VL.
Share your ideas and vote on what matters to help shape the future of Nemotron.
Stay up to date on NVIDIA Nemotron by subscribing to NVIDIA news and following NVIDIA AI on LinkedIn, X, YouTube, and the Nemotron channel on Discord.
Related Posts
TML Inkling on vLLM: Day-0 Support with Optimized Performance
vLLM brings day-0 support to TML Inkling, a 1T-parameter multimodal model, with MTP, long-context serving, parallelism, and up to 380 tokens per second per user on NVIDIA GB200 GPUs.