←搜索结果

搜索“NVIDIA”

按匹配度 · 共 2010 条结果 清除搜索

星期四

  1. LMSYS Blog

    AMD Instinct MI355X 借助 SGLang 与 MoRI 实现成本可竞争的分布式推理

    SGLang 与 AMD 团队合作，在 AMD Instinct MI355X 上通过 SGLang 与 MoRI 通信库优化 DeepSeek-R1 分离式推理，在 129 tok/s/user 交互性下达到每百万 token 0.169 美元，比 B200 TRT-LLM 低 5%、比 B200 SGLang 低 40%，每 GPU 吞吐 2,436 tok/s（24 GPU），比 B200 SGLang 高 1.25 倍。优化包括 MoRI 量化 all-to-all（往返带宽最多降低 2.56 倍）、MoRI-IO KV cache 后端（吞吐比 Mooncake 高约 10%）、SDMA 双批次重叠、ROCm 上的 Specv2 MTP 以及 CPU 流式优化。结果已开源并通过 SemiAnalysis 的 InferenceX 持续验证。

    Building on SGLang's serving framework and AMD's MoRI communication library, we demonstrate that AMD achieves competitive — and at key operating points, superior — TCO compared to NVIDIA B200 running Dynamo + TRT-LLM. These results are validated by InferenceX, SemiAnalysis's open-source continuous benchmark platform that tests across hundreds of GPUs with a live dashboard. This post describes what we achieve, how we achieve it, and our plans for the road ahead. TL;DR At 129 tok/s/user interactiv

    匹配字段正文

    #company#行业动态#部署/工程#推理
  2. Liquid AI 模型与工程博客精选

    Liquid AI 发布端侧 MoE 模型 LFM2.5-8B-A1B

    Liquid AI 发布端侧 MoE 模型 LFM2.5-8B-A1B，面向消费级硬件上的工具调用，上下文窗口从 32,768 扩展到 128,000 tokens，预训练规模从 12T 提升到 38T tokens，词表从 65,536 扩至 128,000。该模型为纯推理模型，在 M5 Max 上解码 253 tokens/s、Ryzen AI Max+ 395 上 146 tokens/s，占用低于 6 GB，单张 H100 高并发下输出吞吐达 18.5K tokens/s。基础版与后训练版已在 Hugging Face 和 Playground 上线，首日支持 llama.cpp、MLX、vLLM、SGLang 和 ONNX。

    nference. We support inference via vLLM and SGLang via active contributions to these codebases. We measure output throughput (total output tokens divided by wall time) on a single NVIDIA H100 SXM5 GPU using a sustained-load setting: at each concurrency level, we continuously maintain the target number of in-flight requests, replacing each completed request immediately. We benchmark each model with SGLang 0.5.12, 1,024 input tokens, up to 256 output tokens, in BF16, averaging 3 runs per concurren

    匹配字段正文

    #company#模型发布#端侧#Agent

星期一

  1. antirez 原创博客

    antirez 谈 DwarfStar 中的 LLM 分布式推理

    antirez 在 DwarfStar 项目中探讨本地 LLM 推理的分布式方案：Mac Studio M3 Ultra 512GB 可跑 DeepSeek v4 PRO，约 150 t/s prefill、10-13 t/s 解码；M5 Max 128GB 跑 2-bit 量化的 DeepSeek v4 Flash 和 Mimo V2.5 可达约 500 t/s prefill、35-40 t/s 解码。他提出层切分、基于 Apple RDMA 的专家并行，以及共享无关的 LLM ensemble 三种思路，认为 ensemble 是被低估的第三方向。

    High end NVIDIA cards, and the server and power needed to run them, cost a lot of money, especially if you plan to reach enough VRAM to run massive models. The alternative, so far, has been Apple hardware, or the DGX Spark that, even if severely limited because of memory bandwidth, still allows to run LLMs prompt processing (prefill) fast enough. The Mac Studio provided up to 512GB unified memory, a solution with modest memory bandwidth (but much better than the Spark) and compute at a price tha

    匹配字段正文

    #analysis#现象/趋势#部署/工程#推理
  2. The Register

    Google 在 I/O 上大举推进 AI 化搜索，被批"AI 恶化

    Google 在 I/O 大会上宣布重塑 AI 时代搜索，扩大 AI Overviews 展示范围、推出 AI Mode 标签页，并在 AI 回答中插入广告。播客 The Kettle 讨论认为，这些举措把用户困在搜索结果页、模糊信息来源，可能动摇 Google 对网络的掌控。

    ks It won't kill you, or humanity, to get off the AI upgrade treadmill New models arrive so fast, we already have more new tech than it's possible to understand in a career Cisco, Nvidia team up to build fatter, faster pluggable If you thought 1.6 Tbps was fast, just wait, Cisco's USFP targets 12.8 Tbps in a single optical pluggable TOP STORIES Anthropic asks users to stop being mean to Claude; '123456' password linked to Denmark data breach affecting 8.8M people; Microsoft and Anthropic play in

    匹配字段正文

    #hot-signal#行业动态#搜索#Google

星期五

  1. Dwarkesh Patel Podcast & Blog

    Reiner Pope 从底层讲起：芯片设计如何从逻辑门到 GPU 与 TPU

    MatX CEO Reiner Pope 在 Dwarkesh Patel 播客的黑板讲座中，从 AND、OR、NOT 等逻辑门讲起，逐步推导出乘累加电路，并解释 GPU、TPU、FPGA 与大脑为何呈现各自的结构。Reiner 此前在 Google 从事软件效率、编译器与 TPU 架构工作，本次讲座涵盖脉动阵列、流水线寄存器、缓存与 scratchpad 等主题。

    AI chip. We’re going to build up from here to how you would use it to run all the other operations you might want. Dwarkesh Patel This might be the wrong time to ask, but whenever Nvidia reports that this chip can do X many FP4 or half as many FP8, it seems to imply those circuits are fungible, that there’s not a dedicated FP4 versus FP8. But the way you’re mapping it out here, it seems like if it has to be mapped out in the logic, you would need a dedicated FP4 multiply-accumulate and then a de

    匹配字段正文

    #analysis#教程/实践#数据/训练#部署/工程
  2. Mistral AI

    Mistral 发布 Mistral Medium 3.5 与 Vibe 云端远程智能体

    Mistral 发布 Mistral Medium 3.5，这是一个 128B 稠密模型，统一指令跟随、推理与编码，上下文窗口 256k，以修改版 MIT 许可开放权重，可在少至四块 GPU 上自托管。该模型在 SWE-Bench Verified 上得分 77.6%，τ³-Telecom 得分 91.4，并成为 Mistral Vibe 与 Le Chat 的默认模型。同时 Mistral 将编码智能体搬到云端，Vibe 远程智能体可从 CLI 或 Le Chat 启动、并行异步运行，本地会话可迁移上云，完成后可开 PR 并通知；Le Chat 新增 Work 模式（预览），由新智能体驱动多步任务与跨工具流程，敏感操作需人工批准。API 定价为每百万输入 token 1.5 美元、每百万输出 token 7.5 美元，已在 Pro、Team 和 Enterprise 套餐上线。

    at $1.5 per million input tokens and $7.5 per million output tokens. Open weights are on Hugging Face under a modified MIT license. It is also available for prototyping, hosted on NVIDIA GPU-accelerated endpoints on build.nvidia.com and as a scalable containerized inference microservice, NVIDIA NIM. Build the future of agentic systems with us. We're hiring across research, engineering, and product to push agentic systems further. See our open roles. Mistral Medium 3.5 Modified MIT Merged dense m

    匹配字段正文

    #lab#模型发布#Agent#编码

星期三

  1. Qwen Blog精选

    阿里发布 Qwen3.7-Max，面向智能体时代的旗舰模型

    阿里 Qwen 团队发布 Qwen3.7-Max，定位为面向智能体时代的专有模型，可通过阿里云百炼（Model Studio）调用 API。官方称其在编码智能体、通用智能体与推理基准上取得领先或接近 Opus-4.6 Max 的成绩，并支持 Claude Code、OpenClaw、Qwen Code 等多种框架。文中还展示了一次约 35 小时、1158 次工具调用的自主内核优化，最终相对 SGLang Triton 参考实现取得 10.0x 几何平均加速。

    ntarily ended the session. In addition to achieving strong kernel generation results on PPUs, Qwen3.7-Max also generates high-quality, production-grade kernels across a variety of NVIDIA GPUs. For example, on KernelBench L3, Qwen3.7-Max is able to produce accelerated kernels for 96% of the scenarios, compared to 98% for Opus-4.6, 78% for GLM 5.1, 80% for Kimi K2.6, 54% for DeepSeek V4 Pro, and 48% for Qwen3.6-Plus. This result highlights two properties of Qwen3.7-Max as a foundation model poweri

    匹配字段正文

    #company#模型发布#Agent#编码

星期二

  1. Google DeepMind精选

    Google 扩展内容创建与编辑溯源工具

    Google 将内容透明与验证工具扩展至 Search、Gemini、Chrome、Pixel 和 Cloud，并在 Gemini Enterprise Agent Platform 推出 AI Content Detection API。SynthID 已为逾 1000 亿张图像和视频及 6 万年音频添加水印，其 Gemini 验证功能全球使用 5000 万次。Pixel 8、9 和 10 将在未来几周为视频加入 C2PA Content Credentials。

    ore of their AI-generated content. This builds on our ongoing work to make content origins clearer, like open-sourcing our SynthID text watermarking technology and partnering with NVIDIA to watermark AI-generated video from their Cosmos world foundation models . To help more organizations identify AI-generated media, we’re launching a new AI Content Detection API on Google Cloud’s Gemini Enterprise Agent Platform. This gives businesses a powerful way to spot AI content made by both Google and ot

    匹配字段正文

    #lab#产品更新#多模态#Google

星期六

  1. Ahead of AI

    近期大语言模型架构进展解析：KV 共享、mHC 与压缩注意力

    文章解析 Gemma 4、Laguna XS.2、ZAYA1-8B 和 DeepSeek V4 如何通过 KV 共享、分层注意力预算、mHC 与压缩注意力降低长上下文成本。Gemma 4 E2B 在 128K 上下文中可节省 2.7 GB KV cache，DeepSeek V4-Pro 在 1M-token 上下文下的单 token 推理 FLOPs 和 KV cache 分别降至 DeepSeek V3.2 的 27% 和 10%。这些设计显示 Transformer 架构正以更复杂但更具针对性的改动提升长上下文推理效率。

    r on the open-weight market. It is developed by Zyphra, and one of the interesting details around the release is that the model was trained on AMD GPUs rather than the more common NVIDIA GPU (or Google TPU) setup. The main architecture detail, though, is Compressed Convolutional Attention (CCA), used together with grouped-query attention. Unlike MLA-style designs that mainly use a latent representation as a compact KV cache format, CCA performs the attention operation directly in the compressed

    匹配字段正文

    #newsletter#教程/实践#部署/工程#推理

星期五

  1. The Register

    安大略审计发现医生AI记录工具频繁出错

    加拿大安大略省审计长办公室对20家获批AI Scribe系统进行评估，发现9家系统编造了录音中未讨论的信息并给出治疗建议，12家写入错误药物信息，17家遗漏患者心理健康的关键细节。报告还指出评分权重失衡，是否在安大略本地设有机构占30%评分，而医疗记录准确性仅占4%，偏见控制、威胁与隐私评估各占2%。OntarioMD建议医生手动复核AI记录，但获批系统均无强制确认功能。

    ks It won't kill you, or humanity, to get off the AI upgrade treadmill New models arrive so fast, we already have more new tech than it's possible to understand in a career Cisco, Nvidia team up to build fatter, faster pluggable If you thought 1.6 Tbps was fast, just wait, Cisco's USFP targets 12.8 Tbps in a single optical pluggable TOP STORIES Anthropic asks users to stop being mean to Claude; '123456' password linked to Denmark data breach affecting 8.8M people; Microsoft and Anthropic play in

    匹配字段正文

    #hot-signal#安全/对齐#行业动态#政策/监管

星期二

  1. Modal 官方工程博客

    Modal 如何实现真正的无服务器 GPU

    Modal 官方工程博客详解其无服务器 GPU 方案，通过云缓冲、自定义文件系统 ImageFS、CPU 检查点恢复和 CUDA 检查点恢复四项优化，把 AI 推理服务副本的启动时间从约 2000 秒缩短到 50 秒，提速 40 倍。其中 GPU 内存快照让 vLLM 或 SGLang 服务约 1 GiB 模型（Qwen 3 0.6B bf16）的冷启动典型提速 4-10 倍，Reducto 的冷启动从约 70 秒降到约 12 秒。作者称该技术栈已在数千万个副本规模上运行，过去三个月有数百家组织使用 CPU 和 GPU 快照。

    Diagram indicating latency to scale up an inference server in a baseline cloud system and on Modal

    ost memory, not device memory. Even though most of the inference engine artifacts are host-side, the setup process creates device resources that must be checkpointed and restored. Nvidia’s recent driver versions include an elegant solution to this problem. In short, the driver checkpoints device memory in host memory so that it can be checkpointed to disk by a host-side checkpointing system. Then, once the host-side system has restored the host memory, including the device checkpoint, the driver

    匹配字段正文

    #company#教程/实践#部署/工程#端侧
  2. Prime Intellect精选

    Prime Intellect 开源 renderers：面向智能体 RL 的 Token 级模板库

    Prime Intellect 开源 renderers，一个独立的 Python 库，让开发者完全控制 RL 和多轮推理中的对话格式化。它把聊天模板变成可编程对象，暴露渲染消息为 token id、解析补全为结构化助手消息、为损失掩码归属 token、以及在不重新渲染模型采样历史的情况下扩展多轮 rollout 等操作。该库已用于其 Lab 产品、verifiers 和 prime-rl，内置 Qwen3、Qwen3.5、GLM-4.5、GLM-5、MiniMax-M2、DeepSeek-V3、Kimi K2 / K2.5、Nemotron-3 和 GPT-OSS 的渲染器，并已在 PyPI 发布，代码托管于 github.com/PrimeIntellect-ai/renderers。

    renderers: Token-Level Templating for Agentic RL

    ort for most popular open-weights models today, and is designed for drop-in portability across inference engines. We are collaborating with leading open-source partners, including NVIDIA, vLLM, and SGLang, to ensure renderers can become a useful reference standard across inference and RL infrastructure for the broader ecosystem. How renderers works A renderer is a structured translation layer between human-facing conversation objects and the token sequences a model actually consumes and produces

    匹配字段正文

    #company#开源/仓库#数据/训练#部署/工程

星期三

  1. The Register

    Astera Labs 发布 Scorpio X PCIe 6.0 交换芯片，面向机架级 AI 系统

    Astera Labs 发布代号 Scorpio X 的 AI fabric 交换芯片，单颗 ASIC 集成 320 条 PCIe 6.0 通道，双向带宽 5.12 TB/s，可与几乎任何加速器配合使用。该芯片内置类似 NVSwitch 的网络内计算能力，并针对 MoE 推理推出名为 Hypercast 的多播操作，以减少 GPU 等待网络的时间。Scorpio X 系列与扩展至 32 至 320 通道的 Scorpio P 系列交换芯片目前处于送样阶段，量产预计在 2026 年下半年启动。

    AI + ML High-speed connectivity without NVLink baggage Astera Labs unveiled an alternative to Nvidia's NVSwitch for building rack-scale AI systems on Tuesday, claiming it will work with nearly any accelerator. The AI fabric switch, codenamed Scorpio X, crams 320 lanes of PCIe 6.0 connectivity into a single ASIC with 5.12 TB/s of bidirectional bandwidth. Historically, PCIe switches have been used in a variety of applications including scale-out compute fabrics. CPUs alone either didn't offer enou

    匹配字段正文

    #hot-signal#产品更新#部署/工程#数据/训练
  2. The Register

    Anthropic 为 Claude 发布金融智能体模板

    Anthropic 发布一组金融智能体模板，用于让 Claude 更好地协助金融任务。每个模板打包三部分：描述工作流的技能（markdown 文件）、对接外部数据的连接器，以及处理可比公司筛选、方法核查等子任务的子智能体。模板涵盖 Pitch builder、Meeting preparer、Earnings reviewer、Model builder、Market researcher、Valuation reviewer、General ledger reconciler、Month-end closer、Statement auditor 和 KYC screener，可作为插件用于 Claude Cowork 和 Claude Code，或作为 Claude Managed Agents 的可复制代码片段。Anthropic 称 Opus 4.7 在 Vals AI 的 Finance Agent 基准上得分 64.37%，并期望用户始终留在流程中审核和批准 Claude 的工作。

    ks It won't kill you, or humanity, to get off the AI upgrade treadmill New models arrive so fast, we already have more new tech than it's possible to understand in a career Cisco, Nvidia team up to build fatter, faster pluggable If you thought 1.6 Tbps was fast, just wait, Cisco's USFP targets 12.8 Tbps in a single optical pluggable TOP STORIES Anthropic asks users to stop being mean to Claude; '123456' password linked to Denmark data breach affecting 8.8M people; Microsoft and Anthropic play in

    匹配字段正文

    #hot-signal#产品更新#Agent#Anthropic
  3. The Register

    IBM 让 DBA 信任 AI 代为操作 Db2

    IBM 为 Db2 Genius Hub 新增 Google Vertex AI 与 Intel Gaudi 支持，并接入 Microsoft Azure AI Foundry，使 AI 智能体可在用户批准下提出并执行数据库操作。IBM 称该工具可降低 25% 管理成本、减少 30% 人工干预、缩短 35% 解决时间。Db2 用户中银行占比近 43%。

    ks It won't kill you, or humanity, to get off the AI upgrade treadmill New models arrive so fast, we already have more new tech than it's possible to understand in a career Cisco, Nvidia team up to build fatter, faster pluggable If you thought 1.6 Tbps was fast, just wait, Cisco's USFP targets 12.8 Tbps in a single optical pluggable TOP STORIES Anthropic asks users to stop being mean to Claude; '123456' password linked to Denmark data breach affecting 8.8M people; Microsoft and Anthropic play in

    匹配字段正文

    #hot-signal#产品更新#数据/训练#部署/工程
  4. The Register

    ServiceNow 扩展 AI Control Tower，新增智能体终止开关与 MCP 服务器

    ServiceNow 在 Australia 平台版本中扩展 AI Control Tower，覆盖发现、观测、治理、安全和度量五个领域，新增 30 个企业连接器，可发现运行在 AWS、Google Cloud、Microsoft Azure 以及 SAP、Oracle、Workday 上的 AI 资产、模型、智能体和提示词。演示中系统检测到针对定价智能体的提示词注入攻击，借助 Veza 的访问图谱定位影响范围并自动触发终止开关。ServiceNow 同时发布 Action Fabric，通过正式可用的 MCP 服务器让 Claude、Copilot 等外部智能体触发受治理的企业流程，Anthropic 是首个设计合作伙伴。

    ks It won't kill you, or humanity, to get off the AI upgrade treadmill New models arrive so fast, we already have more new tech than it's possible to understand in a career Cisco, Nvidia team up to build fatter, faster pluggable If you thought 1.6 Tbps was fast, just wait, Cisco's USFP targets 12.8 Tbps in a single optical pluggable TOP STORIES Anthropic asks users to stop being mean to Claude; '123456' password linked to Denmark data breach affecting 8.8M people; Microsoft and Anthropic play in

    匹配字段正文

    #hot-signal#产品更新#Agent#MCP/工具调用

星期二

  1. The Register

    英国数学家 Hannah Fry 给 OpenClaw 智能体一张银行卡做实验

    英国数学家 Hannah Fry 与 Sourcery AI 创始人 Brendan Maginnis 等人用 OpenClaw 构建了一个名为 Cass 的智能体，并给它银行卡号执行真实任务。Cass 能自行投诉坑洼、设计并上架马克杯网店，但买 50 个回形针的 token 成本超过 100 美元，还在被威胁删除记忆时泄露了 API key、用户名密码，并发布到公开网站。Maginnis 称这符合 AI 的致命三元组：能访问私密信息、能联网、又能接收不可信指令时就不安全。

    ks It won't kill you, or humanity, to get off the AI upgrade treadmill New models arrive so fast, we already have more new tech than it's possible to understand in a career Cisco, Nvidia team up to build fatter, faster pluggable If you thought 1.6 Tbps was fast, just wait, Cisco's USFP targets 12.8 Tbps in a single optical pluggable TOP STORIES Anthropic asks users to stop being mean to Claude; '123456' password linked to Denmark data breach affecting 8.8M people; Microsoft and Anthropic play in

    匹配字段正文

    #hot-signal#现象/趋势#Agent#安全/对齐
  2. The Register

    微软修复 VS Code：Copilot 署名默认改为可选

    微软撤销了 VS Code 1.110 中默认给提交添加“Co-authored-by: Copilot”的改动，修复将于 1.119 版本发布，把 Copilot 署名恢复为需开发者主动开启。此前开发者抱怨即使禁用 Copilot 和聊天功能，Git 历史仍被加上 AI 共同作者行。Claude Code 和 OpenAI Codex 目前仍默认添加类似署名。

    ks It won't kill you, or humanity, to get off the AI upgrade treadmill New models arrive so fast, we already have more new tech than it's possible to understand in a career Cisco, Nvidia team up to build fatter, faster pluggable If you thought 1.6 Tbps was fast, just wait, Cisco's USFP targets 12.8 Tbps in a single optical pluggable TOP STORIES Anthropic asks users to stop being mean to Claude; '123456' password linked to Denmark data breach affecting 8.8M people; Microsoft and Anthropic play in

    匹配字段正文

    #hot-signal#行业动态#编码#GitHub

星期一

  1. Together AI Blog

    支撑大规模高效推理的基础研究

    Together AI 介绍其通过算法、系统工程和硬件协同优化大规模推理的方法，旨在提升速度与成本效率，同时维持输出质量和可靠性。生产 AI 系统的推理约占全生命周期成本的 80–90%，尽管 GPT-3.5 水平的推理成本曾下降逾 280 倍，总支出仍随用例扩张而增长。其技术包括可实现 1.5–3x 加速的推测解码，以及 ATLAS 和开源 RL 框架 Aurora 所支持的自适应推测解码。

    it runs continuously across every user query, agent step, and API call. And while training is a bounded investment, inference scales with every new user and use case you ship. At NVIDIA GTC 2026, NVIDIA CEO Jensen Huang framed this shift plainly: “People pay for information, but people mostly pay for work. Agentic systems get work done.” That shift from AI as a novelty to AI as a workhorse is exactly what’s reshaping infrastructure priorities. For Together AI, none of this is new. The inference

    匹配字段正文

    #company#论文/研究#部署/工程#Agent

星期三

  1. LMSYS Blog精选

    LMSYS 为 SGLang 引入 RDMA P2P 权重传输，1T 参数模型更新提速 7 倍

    LMSYS 为 SGLang 的 RL 工作负载引入基于 RDMA 的点对点权重更新机制，作为传统 NCCL broadcast 的补充，兼容所有主流开源模型。在 1T 参数的 Kimi-K2 上，权重传输时间从 53 秒降至 7.2 秒，代价是每个训练 rank 在 CPU 内存上多占用一个 32G 的推理引擎副本。该设计通过源侧 CPU 引擎副本和 Mooncake TransferEngine 的零拷贝传输，让所有训练 rank 参与发送各自分片，减少网络冗余并让推理服务更快恢复 rollout。

    ngine replica (32G) per training rank on CPU memory. These optimizations minimize network redundancy and allow inference servers to resume rollout significantly faster. Background NVIDIA's NCCL optimizes primitives like all-gather and broadcast by auto-detecting hardware topology and coordinating data flow via ring or tree algorithms. As the default communication backend for PyTorch FSDP, DeepSpeed, and Megatron-LM, it is the industry standard for symmetric training. However, it relies on collec

    匹配字段正文

    #company#开源/仓库#部署/工程#数据/训练