AI Trends

Trending story

LMSYS Optimizes Disaggregated Serving of DeepSeek-V4.1-Flash on H20-3e

1 reports1 story updatesUpdated Nov 19 08:00

Understand this story

AI overview

On November 19, 2026, LMSYS announced optimizations for disaggregated prefill and decoding serving of DeepSeek-V4.1-Flash on H20-3e. Compared with the TP4 baseline, average end-to-end latency decreased by 65.5%, while output throughput rose to 2.91 times the baseline; the final configuration reduced average latency from 13.63 seconds to 4.70 seconds and increased output throughput from 847 to 2,465 tokens/s. LMSYS said the main gains came from prefill CP, decoding DP, and FP8 DeepGEMM, and also provided colocated serving results, item-by-item ablation data, and SGLang-based reproduction commands. There are currently no other substantiated developments.

AI-generated from public reports

Latest development · Report

LMSYS Analyzes PD and Co-located Serving Optimizations for DeepSeek-V4.1-Flash on H20-3e

Developments

1 confirmed developments

  1. LMSYS Optimizes Disaggregated Serving for DeepSeek-V4.1-Flash

    Source:LMSYS Blog

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 19

  1. LMSYS BlogFirst-party

    LMSYS Analyzes PD and Co-located Serving Optimizations for DeepSeek-V4.1-Flash on H20-3e

    LMSYS optimized disaggregated prefill and decoding serving for DeepSeek-V4.1-Flash on H20-3e, reducing average end-to-end latency by 65.5% and increasing output throughput to 2.91× that of the TP4 baseline. The final configuration reduced average latency from 13.63 s to 4.70 s and increased output throughput from 847 to 2,465 tokens/s, with the main gains coming from prefill CP, decoding DP, and FP8 DeepGEMM. The article also presents co-located serving results, step-by-step ablation data, and SGLang-based reproduction commands.

Related stories

  1. Related storiesOpenAI Releases GPT-6.2 Astra Model
  2. Related storiesDeepSeek releases DeepSeek-V4.1-Flash-1017
  3. Related storiesAnthropic releases Claude Opus 5.5
  4. Related storiesEnglish translation is being prepared.