←搜索结果

按匹配度 · 共 297 条结果 清除搜索

星期五

  1. LMSYS Blog

    腾讯混元 HPC-Ops 的 Attention、Router GEMM 与 MoE 算子并入 SGLang 主分支

    腾讯混元开源算子库 HPC-Ops 的 Attention、Router GEMM 和 MoE 三个算子已并入 SGLang 主分支,面向 NVIDIA Hopper(SM90)GPU,并已用 Qwen3、Hy3 和 LongCat 工作负载验证。在 8× H20 上以 Hy3-FP8 同时启用 Attention 与 MoE,batch size 4–64 时 TPOT 降低 15.1–48.8%,batch size 4–16 时 TTFT 降低 3.3–6.0%;LongCat-Flash-Lite-FP8 启用 Router GEMM 后输入吞吐提升 5.5–6.1%。这些算子已在腾讯大规模生产服务中部署,其中 Dynamic Attention 与 Fused MoE 使 Hy3 模型的 TPOT 最多降低 48.8%。

    EMM produces the scores used for top-k selection, where small numerical changes can alter expert choices. The selected experts then process small and uneven token groups, allowing metadata construction, token movement, intermediate storage, and launch overhead to rival the expert GEMMs themselves. HPC-Ops addresses each stage with a dedicated operator: workload-aware scheduling for Attention, a precision-aware formulation for Router GEMM, and a fused pipeline for MoE that eliminates the standalo

    匹配字段正文

星期二

  1. LMSYS Blog

    LMSYS 发布 SpecForge v0.3.0:分离式与同置式推测解码训练栈及新 SpecBundle 草稿模型

    LMSYS 发布 SpecForge v0.3.0,将目标模型推理与草稿模型训练解耦为生产者与消费者两个资源池,在 8×H20 上以 3 台 SGLang 捕获服务器加 5 个训练 worker 的拓扑,端到端训练吞吐较此前同置实现提升约 10%。新运行时统一支持 EAGLE3、EAGLE3.1、P-EAGLE、DFlash、Domino 和 DSpark 多种推测解码方法,并新增训练与推理一致性校验流程。同时发布 SpecBundle 草稿模型集合,11 个 checkpoint 中有 9 个来自蚂蚁集团 AQ、RadixArk、招商银行、Domino 作者及社区成员,全部仅用开放数据训练。

    SpecForge online-disaggregated training architecture

    and a consumer pool. Producers schedule prompts across patched SGLang capture servers. SGLang writes feature tensors to Mooncake, while SpecForge sends only lightweight SampleRef metadata through the control plane. Trainer ranks resolve those references when they are ready to build a batch. Figure 1. The online-disaggregated training flow. Large tensors remain in the data plane; references and lifecycle state travel through the control plane. 1. The capture contract Every URL in deployment.disa

    匹配字段正文

星期四

  1. Modal 官方工程博客精选

    Modal 重建沙箱平台,可并发运行 100 万个沙箱

    Modal 重建了核心沙箱平台,可并发运行数百万个沙箱、每秒创建数万个沙箱,并演示了在不到一分钟内创建全部 100 万个沙箱。新架构去掉了控制面的中心协调,每个 worker 自持状态并通过 Redis stream 异步发布,沙箱创建路径只需两次网络跳转,中位启动时间不到半秒。该系统已进入 Beta,用户可通过一处代码改动选择启用。

    Evidence that we can run a lot of sandboxes.

    rces, or otherwise reject it; We have no data stores in the critical path of sandbox creation at all, which improves scalability and reliability. While we do need to write sandbox metadata and results to durable storage, we do so largely asynchronously; Other than sandbox creations, we have no RPCs which are O(sandboxes). Workers batch control messages for multiple sandboxes together in single RPCs, in the spirit of ideas from data-oriented design The result is that the sandbox creation path req

    匹配字段正文

星期三

  1. LMSYS Blog精选

    SGLang 与 Miles 为多模态模型 Inkling 提供 Day-0 支持

    LMSYS 与 Thinking Machines 团队合作,为 975B 参数多模态模型 Inkling 在 SGLang 和 Miles 上提供 Day-0 支持。SGLang 针对其短卷积、相对位置注意力和共享专家 sink 的 MoE 架构做了专门优化,在 Nvidia Blackwell GPU 上实现最高 71.7k tok/s 输入吞吐和 171.0 tok/s 单用户解码速度,并支持投机解码、PD 分离、多 LoRA 服务与 HiCache。Miles 通过定制 Megatron 后端支持全参数与 LoRA 强化学习,覆盖文本和视觉语言任务。

    apture most of the forward pass once, then replay it without re-dispatching Python each step. Both still have to fall back to eager execution wherever an op needs live per-request metadata, and Inkling has four ShortConv sites per layer doing exactly that, on top of attention's own. Each site's eager fallback is a real, multi-level Python call stack, not a single kernel launch: Figure 2: Under BCG, each of Inkling's four ShortConv sites (K stream, V stream, attention-output, MLP/MoE-output) fall

    匹配字段正文

星期二

  1. Hugging Face精选

    Microsoft Foundry 托管计算上的 Hugging Face 模型

    Microsoft 与 Hugging Face 将经过许可与安全审查的数千个开放模型引入 Foundry Model Catalog,目前以预览版提供,并每周更新。模型可通过统一端点部署到配备 NVIDIA A100、H100 或 AMD MI300X 的 Managed Compute,运行时升级和 CVE 修补由平台自动完成;后续将扩展模型与加速器覆盖,并支持 Bring Your Own Weights。

    t Foundry is a platform for building and operating agentic AI applications. Foundry starts with the widest model selection on any cloud — models from Microsoft, OpenAI, Anthropic, Meta, Mistral, DeepSeek, Hugging Face, and others, spanning frontier, open-source, and custom weights — all accessible through a single endpoint and a single set of SDKs in Python, C , JavaScript, and Java. On top of those models sits the Foundry Agent Service : multi-agent orchestration with built-in memory, knowledge

    匹配字段正文

  2. Hugging Face精选

    LeRobot v0.6.0:想象、评估、改进

    Hugging Face 发布 LeRobot v0.6.0,引入三种世界模型策略、多款新 VLA、统一奖励模型 API,以及由 lerobot-eval 支持的六个新仿真基准。新版还增加支持 DAgger 人工纠正的 lerobot-rollout、FSDP 与 HF Jobs 云训练,并为数据集加入深度数据、自动语言标注、自定义视频编码及最高约 2 倍的加载提速。

    encies with their own install steps; each docs page has the exact recipe, and every benchmark ships a ready-made Docker image if you'd rather skip the setup. Together with LIBERO, Meta-World, and NVIDIA IsaacLab-Arena, that makes nine benchmark families under one roof, and a new Adding a New Benchmark guide documents exactly how to plug in yours. Evaluation also got faster: parallel eval now defaults to async vectorized environments, benchmarked at up to 2x faster. Training & inference lerobot-r

    匹配字段正文

星期一

  1. Hugging Face精选

    Hugging Face Kernels 重大更新:新增 Hub 仓库类型并强化安全机制

    Hugging Face 为 Kernels 新增 Hub 仓库类型,并通过可信发布者、可复现构建和 Sigstore cosign 代码签名强化安全;签名加载时自动验证尚未启用。更新还重构了 kernels 与 kernel-builder CLI,支持 Torch Stable ABI 和 Apache TVM FFI,并为智能体构建、基准测试和迭代优化 kernel 提供基础。

    128-x86 64-windows: rejected (CPU (x86 64) does not match system CPU (aarch64)) torch211-cu128-x86 64-windows: rejected (CPU (x86 64) does not match system CPU (aarch64)) torch212-metal-aarch64-darwin: rejected (OS (darwin) does not match system OS (linux)) torch211-metal-aarch64-darwin: rejected (OS (darwin) does not match system OS (linux)) torch210-metal-aarch64-darwin: rejected (OS (darwin) does not match system OS (linux)) torch29-metal-aarch64-darwin: rejected (OS (darwin) does not match s

    匹配字段正文

  2. LMSYS Blog精选

    SGLang 集成 DSpark 投机解码,支持 Qwen3 与 DeepSeek-V4

    LMSYS 在 SGLang 中集成 DSpark 投机解码,支持 Qwen3 等稠密模型和 DeepSeek-V4 等稀疏模型,相关实现见 sgl-project/sglang#30261。DSpark 用半自回归块草稿器加置信度头,按每个请求的置信度动态决定验证长度,避免验证不太可能被接受的 token。官方在 DeepSeek-V4-Flash、H200、DP4 上对比非投机与 MTP 基线,并给出 ragged verify 下的完整 CUDA graph、零开销调度和成本表分析等工程细节。

    Aggregate throughput vs. per-user decode speed on H200 dp4, one curve per arm: non-spec floor, MTP, and DSpark. Right-and-up is better; each marker is a batch size averaged over three rounds.

    reuses attention kernels the backend already has — on DeepSeek-V4 the model's own sparse-MLA path (flash mla), with no new kernel; each supported backend just rebuilds its varlen metadata from the packed layout on graph replay. Figure 2. Fitting a batch with per-request-variable verify lengths into a captured CUDA graph. A fixed-shape graph pads every request to the full block width (N x W); the ragged path front-packs the scheduled tokens and rounds only the total up to the nearest captured ti

    匹配字段正文

星期四

  1. Cognition 模型 / Devin 博客精选

    Cognition 发布 Devin Security Swarm,自动验证漏洞并提交修复 PR

    Cognition 发布 Devin Security Swarm,由并行智能体分段排查代码库,组合出完整攻击路径并在隔离沙箱中复现验证可利用性,确认后自动编写补丁并提交 PR 供审查。官方称其在 50 个真实漏洞基准上比最接近的同类方案多发现漏洞且成本低 30%,并独有发现 PHP 模板注入沙箱绕过等三个关键漏洞。该产品即日起可用,支持从威胁模型文档生成扫描配置、按日/周/自定义计划扫描,首次全量扫描后仅处理变更代码。

    r, Erlang, C, Kotlin, and Dart. Only Devin found three critical vulnerabilities that other tools missed: a PHP sandbox bypass via template injection, an argument injection through metadata value parsing, and an overly broad deserialization surface in Spring Kafka. Scan profiles Devin can generate scan profiles directly from your existing threat model documentation, tailor them to specific attacker personas, and apply them across your entire organization without per-repo configuration or CI setup

    匹配字段正文

星期三

  1. Figma AI / Agent / Make

    Figma 设计智能体新增自定义工具与更强上下文能力

    Figma 宣布其设计智能体新增自定义工具与更强上下文能力,用户可通过提示词在画布上生成可复用的生成式插件和基于 WebGPU 的 shader 效果与填充。智能体支持直接附加文件、引用其他 Figma 文件结构、联网搜索,并通过 MCP 连接器接入 GitHub、Atlassian、Slack 等工具;用户还可把常用提示词保存为自定义技能并发布给团队。使用他人插件或 shader 在所有套餐免费,提示智能体构建插件或 shader 需设计智能体权限,正式可用后将消耗 AI credits。

    Mockup of a mobile design canvas in Figma surrounded by AI-generated task prompts. A poster titled “MODULAR LIVING” by Camille Vosk is selected on the canvas while blue status bubbles display messages such as “Creating QR code plugin,” “Generating code layer,” “Creating custom shader,” and “Checking color contrast,” illustrating AI-assisted design workflows.

    me.” Shader effects and fills Customer snapshot: Anna Zhang Creative technologist Anna Zhang built her own shaders to remix her own photography with collage, marbling, light leak, metal emboss, and prism effects. “It’s a way to compound what I’ve been doing in the past or build a reusable workflow,” she says. “There’s a lot of expressive potential from a single starting point.” Anna focused on the functionality while letting the agent handle the UI: “It was a back-and-forth dialogue between me a

    匹配字段正文

星期三

  1. Transluce

    Transluce 推出 Analysis Plans,为 AI 行为分析提供可验证框架

    Transluce 发布 Analysis Plans,一套用于可验证分析 AI 行为的框架,通过 Python API 编写、在网页界面中逐步审阅。它包含查询步骤(用 DQL 过滤、分组、连接数据)和阅读步骤(用 LLM 分析数据并给出带引用的文本摘要或结构化判断),两类步骤可组合成分析流水线。团队用 Docent 在约 15 分钟内检测 SWE-rebench 上的作弊行为,并演示了识别失败模式、对比 GPT-5.1 Codex 与 GPT-5 Codex 在 Terminal-Bench 上 6.5 个百分点差距等案例。该功能现已向所有 Docent 用户开放。

    Without analysis plans, it's difficult to understand and verify the behavior of a coding agent.

    d analysis before waiting for LLM calls to complete. 2. Anatomy of an analysis plan This analysis plan starts with a DQL query to select successful runs by filtering the agent run metadata to resolved=1. After that, it passes the results to a reading step, which scores runs for cheating. A reading step has the following components: A prompt template with parameters. To detect cheating, we create a rubric that defines suspiciousness, taking one agent run as a parameter. The Docent UI shows the fu

    匹配字段正文

星期一

  1. LMSYS Blog

    LMSYS 发布 DFlash 与 Spec V2 推测解码引擎

    LMSYS 联合 Z Lab 和 Modal 发布 DFlash 推测解码模型与 SGLang 默认的 Spec V2 引擎,DFlash 用块扩散并行起草并通过 KV 注入提升接受长度。在 HumanEval 数据集并发为 1 时,Qwen 3.5 397B-A17B 的 DFlash 模型吞吐达到基线的 4.3 倍以上、MTP 的 1.5 倍。该模型已在 z-lab、modal-labs 和 lmsys 三个 Hugging Face 组织发布,可在 SGLang 中直接部署。

    erlap scheduler. In particular, there are two key opportunities for overlap: host-side pop and process cleanup after the GPU finishes batch N-1 (e.g. stop token detection, request metadata updates) can overlap with GPU work on batch N; host KV allocation (in prepare for decode) for batch N can overlap with GPU work on batch N-1 Under V2 with these optimizations, performance improved by over 33%, from 11.4 ktok/s to 15.3 ktok/s, when running Qwen 3-8B on a single B200 at concurrency 32 (details h

    匹配字段正文

星期四

  1. Mistral AI

    Mistral AI 发布 Search Toolkit 公开预览版

    Mistral AI 发布 Search Toolkit 公开预览版,这是一个用于构建生产级搜索管线的可组合框架,把数据接入、检索和评估整合到共享接口下,开源并支持云端、本地和边缘部署。框架内置 BM25 稀疏检索、稠密向量检索和混合配置,并提供 recall、precision、MRR、NDCG 等评估指标,可独立衡量检索质量。官方称其已在金融服务、制造、公共部门和媒体娱乐等领域使用,CMA CGM 用它配合 Voxtral 处理三路音频数据,端到端 15 秒内返回告警。

    problem. They have a dozen search problems. Internal wikis, support ticket systems, document repositories, file storage, codebases. Each source has different structure, different metadata, and needs different processing to index well. Teams typically end up building a separate ingestion pipeline for each one, with its own parsing logic, its own chunking strategy, and its own assumptions about what a "document" looks like. The result is a set of isolated indexes that can't be searched together,

    匹配字段正文

星期二

  1. Google DeepMind精选

    Google 扩展内容创建与编辑溯源工具

    Google 将内容透明与验证工具扩展至 Search、Gemini、Chrome、Pixel 和 Cloud,并在 Gemini Enterprise Agent Platform 推出 AI Content Detection API。SynthID 已为逾 1000 亿张图像和视频及 6 万年音频添加水印,其 Gemini 验证功能全球使用 5000 万次。Pixel 8、9 和 10 将在未来几周为视频加入 C2PA Content Credentials。

    ology as a member of the C2PA steering committee . This ensures that transparency tools built into our devices work seamlessly across the platforms you use every day. For example, Meta — a fellow C2PA Steering Committee member — will start labeling camera-captured media with Content Credentials on Instagram. This means authentic photos and videos shot natively on Pixel phones will soon be recognized and labeled as such when you share them on Instagram. We’ve long invested in providing helpful co

    匹配字段正文

星期一

  1. DevClass

    Ruby 之父 Matz 借助 Claude Code 开发原生编译器 Spinel

    Ruby 发明者松本行弘(Matz)正在开发原生编译器 Spinel,借助 Anthropic 的 Claude Code 完成,项目已在 GitHub 以 MIT 许可证开源。Spinel 将 Ruby 代码解析为 AST 后转换为 C 代码,再由 gcc 或 Clang 编译为独立原生可执行文件,测试中编译后代码比 MiniRuby 快约 11.6 倍。它目前仅支持 Ruby 子集,不支持 eval、线程、非 UTF-8 编码和运行时元编程,也无法运行 Ruby on Rails 等多数现有代码,仓库中多数代码带有 Claude Open 4.7(1M context)共同作者注释。

    is that it supports only a subset of Ruby. Unsupported features include eval statements, which evaluate and execute Ruby code at runtime, threads, text encoding other than UTF-8, metaprogramming such as defining a method at runtime, and deeply nested lambda functions. Ruby variables are not typed, though objects are strongly typed. Spinel performs type inference to enable C code generation, since C is a strongly typed language. The code makes use of an existing and mature Ruby parser called Pri

    匹配字段正文

星期三

  1. The Register

    Astera Labs 发布 Scorpio X PCIe 6.0 交换芯片,面向机架级 AI 系统

    Astera Labs 发布代号 Scorpio X 的 AI fabric 交换芯片,单颗 ASIC 集成 320 条 PCIe 6.0 通道,双向带宽 5.12 TB/s,可与几乎任何加速器配合使用。该芯片内置类似 NVSwitch 的网络内计算能力,并针对 MoE 推理推出名为 Hypercast 的多播操作,以减少 GPU 等待网络的时间。Scorpio X 系列与扩展至 32 至 320 通道的 Scorpio P 系列交换芯片目前处于送样阶段,量产预计在 2026 年下半年启动。

    disappear; Bandwidth hogs rejoice, Celestica's latest switch is bristling with 64 ports of 1.6 Tbps Ethernet; Google to sell its TPUs to some customers, who also fancy big-G GPUs; Meta Arms itself to the teeth by signing for 'tens of millions' of AWS Graviton cores Alongside its Scorpio X family of chips, Astera is also expanding its Scorpio P-series switches with models ranging from 32 to 320 lanes of PCIe connectivity. All of these switches work with its COSMOS management suite, a hardware mon

    匹配字段正文

星期六

  1. LMSYS Blog精选

    SGLang 与 Miles 为 DeepSeek-V4 提供 Day-0 推理与 RL 训练支持

    LMSYS 宣布 SGLang 与 Miles 在 DeepSeek-V4 发布当天提供推理与 RL 训练支持,覆盖 1.6T Pro 和 284B Flash 两个版本。针对 DeepSeek-V4 的混合稀疏注意力、mHC 和 FP4 专家权重,团队引入 ShadowRadix 前缀缓存、HiSparse CPU 扩展 KV、MTP 投机解码等机制,并集成 FlashMLA、FlashInfer TRTLLM-Gen MoE、DeepGEMM Mega MoE 等内核。RL 侧 Miles 支持 DP/TP/SP/EP/PP/CP 六种并行策略与 FP8 训练,已在 32 张 GB300 上完成 285B 模型训练,首步 rollout 与训练 log-prob 偏差约 0.023。

    DeepSeek-V4 decode throughput on Day-0 launch: SGLang vs other OSS engine

    nference and RL for DeepSeek-V4 (1.6T Pro, 284B Flash). Inference (caching & attention): ShadowRadix prefix cache, HiSparse CPU-extended KV, MTP speculative decoding with in-graph metadata, Flash Compressor, Lightning TopK, hierarchical multi-stream overlap; Inference (kernels & deployment): fast kernel integrations (FlashMLA, FlashInfer TRTLLM-Gen MoE, DeepGEMM Mega MoE, TileLang mHC), DP/TP/CP attention, EP MoE on DeepEP, PD disaggregation; RL training: full parallelism (DP/TP/SP/EP/PP/CP), ti

    匹配字段正文

星期四

  1. DevClass

    Salesforce 在 TDX 发布 Headless 360,让智能体接管平台开发

    Salesforce 在旧金山开发者大会 TDX 上发布 Headless 360,将平台上的 CRM、客服、营销和电商等能力全部变成 API、MCP server 或 CLI 命令,供 Claude Code、Codex、Windsurf、Visual Studio Code 等工具中的智能体调用。同时推出基于 Visual Studio Code 的浏览器 IDE Agentforce Vibes,默认使用 Claude Sonnet 4.5,含 plan 和 act 两种模式及预置智能体技能,并随免费 Salesforce Developer Edition 提供,开发者版限额为每月 110 次请求和 150 万 token,刷新至 5 月 31 日。Salesforce 还宣布将智能体定义语言 Agent Script 开源,并发布 Slack Agent Kit。

    re are new agentic integrations and it reflects the fact that some users will rely on agents to do the coding for them. Salesforce extensions, the Salesforce CLI, and organization metadata are all pre-configured. The default LLM (large language model) in Agentforce Vibes is Claude Sonnet 4.5. It has two modes - plan mode and act mode - and comes with a set of pre-defined agent skills for tasks such as creating custom tabs in a UI (user interface) or generating Salesforce flows. Despite the flexi

    匹配字段正文

星期三

  1. Anthropic Engineering

    Anthropic 推出 Managed Agents,将智能体的大脑与手解耦

    Anthropic 在 Claude Platform 上推出托管服务 Managed Agents,把智能体拆分为 session、harness 和 sandbox 三个可独立替换的接口,让 Claude 与执行环境解耦。解耦后容器按需通过工具调用创建,官方称 p50 首 token 延迟下降约 60%、p95 下降超过 90%。凭证不再进入运行 Claude 生成代码的沙箱,Git 令牌在沙箱初始化时注入,MCP 的 OAuth token 存放在 vault 中并由代理调用。

    grams that didn't exist yet. With Managed Agents, we aimed to design a system that accommodates future harnesses, sandboxes, or other components around Claude. Managed Agents is a meta-harness in the same spirit, unopinionated about the specific harness that Claude will need in the future. Rather, it is a system with general interfaces that allow many different harnesses. For example, Claude Code is an excellent harness that we use widely across tasks. We’ve also shown that task-specific agent h

    匹配字段正文

星期二

  1. LMSYS Blog

    LMSYS 为 Miles RL 框架提供 ROCm 支持,可在 AMD Instinct GPU 上运行

    LMSYS 宣布开源 RL 框架 Miles 支持 ROCm,可在 AMD Instinct MI300/350 系列 GPU 上原生运行分布式 rollout 与 GRPO 训练。Miles 基于 SGLang 与 Slime 生态,将 rollout 生成与 Megatron 模型优化解耦,并提供预装 SGLang 和 Megatron-LM 的 ROCm 容器。在单节点 8 卡 MI300X 上训练 Qwen3-30B-A3B 时,平均单步耗时 388.50s,其中 rollout 生成占 152.79s,AIME 准确率从 0.665(step 19)升至 0.729(step 139)。

    four smallest numbers are indeed 39, 48, 57, and 66, and their sum is 210. Therefore, the answer is 210. Answer: \boxed{210} python output (no additional tool output in this turn) meta: label=210, reward=1.0 This trajectory shows both current capability and current gap: The model uses tools correctly in turn 1 to derive key intermediate values; In turn 2, it still makes a tool-call schema mistake (code passed directly instead of arguments.code); Turn 3, it fails to correct code and resorts to ju

    匹配字段正文