←搜索结果

搜索“DeepSeek”

按匹配度 · 共 1346 条结果 清除搜索

星期一

  1. vLLM 官方博客精选

    vLLM V1 Alpha 发布，重构核心架构并将吞吐量提升至多 1.7x

    vLLM 发布 V1 Alpha，通过重构调度器、KV cache 管理器、Worker、采样器和 API 服务器，将吞吐量较 V0 提升至多 1.7x。V1 默认启用低开销前缀缓存，增强多模态模型支持，并可通过设置 `VLLM_USE_V1=1` 在不修改现有 API 的情况下启用。当前版本仅支持 Ampere 或更新的 NVIDIA GPU，尚不支持 LoRA、结构化解码、推测解码和流水线并行等功能。

    implementing the speculative decoding support; Kuntai Du is implementing the prefill disaggregation and KV Cache transfer support; Simon Mo and Zhuohan Li contributed to the V1 system design Related Posts DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0 Within three weeks of release, vLLM made DeepSeek-V4.1-Flash 1.9x faster at low concurrency and lifted its throughput 5x on SemiAnalysis AgentX, with SWA bounded replay, CUDA graphs, DeepSeek's new kernels, and vLLM kernel fusions.

    匹配字段正文

    #engineering#产品更新#部署/工程#多模态

星期二

  1. Prime Intellect精选

    Prime Intellect 发布 INTELLECT-MATH 数学推理模型与 500 万条推理轨迹数据集

    Prime Intellect 通过 QwQ-32B 生成并校验合成推理轨迹，微调 Qwen-7B-Math 后再用 PRIME-RL 做在线强化学习，训练出 7B 参数数学推理模型 INTELLECT-MATH，其早期 checkpoint 以 47 个 RL step 达到 Eurus-2-7B-PRIME 用 592 步的效果，训练时间缩短 10 倍。团队同时开源代码、权重和 NuminaMath-QwQ-CoT-5M 数据集，该数据集包含 86 万道数学题上的 500 万条经正确性校验的推理轨迹。文章还提出在推理时计算范式下，RL 中大量前向计算会降低通信需求，使全球分布式训练成为可行路径，并计划今年训练 o3 的完全开源复现。

    Distributed Training in the Inference-Time-Compute Paradigm

    policy initializations for online reinforcement learning. We use this insight to train INTELLECT-MATH, a state-of-the-art model for mathematical reasoning (prior to the release of Deepseek-R1) in the 7B parameter regime that reduces the training time required to match the existing state-of-the-art by 10x. Additionally, we release NuminaMath-QwQ-CoT-5M, a dataset containing five million reasoning traces across 860k mathematics questions verified for correctness. All of our code as well as weights

    匹配字段正文

    #company#模型发布#推理#数据/训练

星期二

  1. vLLM 官方博客

    vLLM 结构化解码入门与 XGrammar 集成解析

    vLLM 介绍结构化解码的原理与实现，并已支持 Outlines 和 XGrammar 两种后端，以约束大语言模型输出符合指定结构。XGrammar 集成在负载下可将每个输出 token 的生成时间改善至多 5x，不适用的请求会回退至 Outlines。规划中的 v1 将重点优化性能，并探索调度器级掩码广播以支持混合请求批处理。

    top level view of structure decoding

    o the same mechanism of "using a format for the model to structurally sampling outputs.” ↩; See this blog post from HuggingFace for using logit processors to control the generation process. ↩ Related Posts DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0 Within three weeks of release, vLLM made DeepSeek-V4.1-Flash 1.9x faster at low concurrency and lifted its throughput 5x on SemiAnalysis AgentX, with SWA bounded replay, CUDA graphs, DeepSeek's new kernels, and vLLM kernel fusions.

    匹配字段正文

    #engineering#教程/实践#部署/工程#Agent

星期四

  1. vLLM 官方博客

    vLLM v0.6.0 发布，吞吐量提升最高 2.7x、TPOT 最多降低 5x

    vLLM 发布 v0.6.0，相比 v0.5.3 吞吐量提升 1.8-2.7x，其中 Llama 8B 的吞吐量提升 2.7x、TPOT 加快 5x，Llama 70B 的吞吐量提升 1.8x、TPOT 减少 2x。主要优化包括拆分 API server 与推理引擎、多步批量调度、异步输出处理、对象缓存和非阻塞数据传输。多步调度目前会造成 token 间延迟波动，并可能在低请求率下提高 TTFT。

    has highest throughput on ShareGPT and Decode-heavy dataset (though the throughput is only marginally higher than TensorRT-LLM), but the throughput of vLLM is lower on Prefill-heavy dataset. Related Posts DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0 Within three weeks of release, vLLM made DeepSeek-V4.1-Flash 1.9x faster at low concurrency and lifted its throughput 5x on SemiAnalysis AgentX, with SWA bounded replay, CUDA graphs, DeepSeek's new kernels, and vLLM kernel fusions.

    匹配字段正文

    #engineering#产品更新#部署/工程#开源生态

星期二

  1. vLLM 官方博客

    vLLM 与 DeepSpeed-FastGen 性能对比分析

    vLLM 在常见场景中速度与 DeepSpeed-FastGen 相当，处理较长输出时更快，其他场景最高可快 1.8x。DeepSpeed-FastGen 仅在长提示词、短输出负载中凭 Dynamic SplitFuse 占优，但实测增益未达其宣称的 2x。

    ing popular features like stop strings and parallel sampling (e.g., beam search). We do expect the DeepSpeed-FastGen is eager to catch up and we welcome the creative innovation in the market! Related Posts DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0 Within three weeks of release, vLLM made DeepSeek-V4.1-Flash 1.9x faster at low concurrency and lifted its throughput 5x on SemiAnalysis AgentX, with SWA bounded replay, CUDA graphs, DeepSeek's new kernels, and vLLM kernel fusions.

    匹配字段正文

    #engineering#评测/基准#部署/工程#开源生态

星期二

  1. vLLM 官方博客精选

    vLLM 发布基于 PagedAttention 的开源 LLM 推理与服务库

    vLLM 发布开源 LLM 推理与服务库，利用 PagedAttention 管理 KV cache，无需修改模型架构即可实现比 HuggingFace Transformers 最高 24x 的吞吐量。实验中其吞吐量比 TGI 最高提升 3.5x，内存浪费降至 4% 以下；LMSYS 部署后将服务所需 GPU 数量减少 50%。

    ire team — Siyuan Zhuang, Ying Sheng, Lianmin Zheng (UC Berkeley), Cody Yu (Independent Researcher), Joey Gonzalez (UC Berkeley), Hao Zhang (UC Berkeley & UCSD), and Ion Stoica (UC Berkeley). Related Posts DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0 Within three weeks of release, vLLM made DeepSeek-V4.1-Flash 1.9x faster at low concurrency and lifted its throughput 5x on SemiAnalysis AgentX, with SWA bounded replay, CUDA graphs, DeepSeek's new kernels, and vLLM kernel fusions.

    匹配字段正文

    #engineering#产品更新#部署/工程#开源生态