AI News

Trending story

NVIDIA blog showcases dual DGX Station deployment of GLM-5.3

1 reports1 story updatesUpdated Nov 14 01:00

Understand this story

AI overview

On November 14, 2026, NVIDIA Developer · Generative AI published a tutorial showing how to deploy GLM-5.3 on two NVIDIA DGX Stations using vLLM. The tutorial connects the two machines via ConnectX-8 RDMA to run the 753B-parameter GLM-5.3 NVFP4-quantized MoE model with tensor parallelism (TP=2); the model requires approximately 464 GB of VRAM, exceeding the 252 GB HBM3e of a single GB300 GPU. The two machines serve externally through a single OpenAI-compatible endpoint and enable multi-token prediction, FP8 KV-cache, and prefix caching. On the SPEED-Bench Coding dataset, AIPerf measured a peak output throughput of 607.8 tokens/s at concurrency 16, dropping to 385.0 tokens/s at concurrency 32, with an average request latency of 453.5 seconds. There are currently no further developments.

AI-generated from public reports

Latest development · Report

One GPU Isn't Enough: Deploying GLM-5.3 on Two NVIDIA DGX Stations with vLLM

Developments

1 confirmed developments

  1. NVIDIA blog demonstrates dual DGX Station deployment of GLM-5.3

    Source:NVIDIA Developer · Generative AI

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 14

  1. NVIDIA Developer · Generative AIFirst-party

    One GPU Isn't Enough: Deploying GLM-5.3 on Two NVIDIA DGX Stations with vLLM

    NVIDIA has released a tutorial demonstrating how to interconnect two DGX Stations via ConnectX-8 RDMA using vLLM to deploy the 753B-parameter GLM-5.3 NVFP4-quantized MoE model with tensor parallelism (TP=2). The model requires approximately 464 GB of VRAM, exceeding the 252 GB HBM3e of a single GB300 GPU. The two machines serve externally through a single OpenAI-compatible endpoint and enable multi-token prediction, FP8 KV-cache, and prefix caching. AIPerf measured a peak output throughput of 607.8 tokens/s at concurrency 16 on the SPEED-Bench Coding dataset, dropping to 385.0 tokens/s at concurrency 32, with an average request latency of 453.5 seconds.

Related stories

  1. Related storiesHugging Face adds decision model support to Transformers
  2. Related storiesEnglish translation is being prepared.
  3. Related storiesi14 migrates its custom architecture to AWS Trainium2
  4. Related storiesAWS introduces vLLM bidirectional KV transfer to reduce multi-turn TTFT