←搜索结果

按匹配度 · 共 264 条结果 清除搜索

星期四

  1. MarkTechPost

    Liquid AI 开源 d1-3B 与 d1-omni-600M 多模态决策模型

    Liquid AI 发布 Open d1 系列两款开源权重多模态决策模型:d1-3B 读取文本与图像,d1-omni-600M 读取文本加图像或音频,两者都不生成文本,而是在一次前向传播中返回校准后的类型化答案,输出 token 数为 0。d1-3B 有 3.12B 参数,基于 LFM2.5-VL-3B,使用 400M SigLIP2 NaFlex 视觉编码器和 32,768 token 上下文;d1-omni-600M 有 587M 参数,基于 LFM2.5-Encoder-350M,音频编码器为 17 层 FastConformer,上下文 16,384 token,音频片段上限 30 秒。两个检查点已上 Hugging Face,可通过 Transformers 加载并支持 llama.cpp,LFM Open License v1.0 允许年收入低于 1000 万美元的免费商用;d1-3B 在 Decision Index v0.2.1 得 48.57,在 RTX 4090 上单问题延迟 8 ms。

    image, or text with audio. Neither model writes text. Each returns calibrated, typed answers in one forward pass with zero output tokens. The target is real-time decisions on the NVIDIA stack: DGX servers, RTX workstations, and Jetson edge boards. Is it deployable? Yes. Both checkpoints are on Hugging Face, load through Transformers, and have day-one llama.cpp support. The LFM Open License v1.0 allows free commercial use below $10 million in annual revenue. d1-omni-600M is an early research rel

    匹配字段正文

星期三

  1. Liquid AI 模型与工程博客精选

    Liquid AI 发布 d1-3B 与 d1-omni-600M 开放权重决策模型

    Liquid AI 发布 d1-3B 和 d1-omni-600M 两款开放权重决策模型,已在 Hugging Face 上线。d1-3B 在 Decision Index v0.2.1 公开集得 48.57,超过所有 10B 以下模型,与 12 倍规模的 Decider 35B-A3B 持平;单次提问在 RTX 4090 上 8 ms、Jetson AGX Thor 上 16 ms、Jetson AGX Orin 上 26 ms、Jetson Orin Nano 上 50 ms。d1-omni-600M 是首个实验性 checkpoint,支持文本与图像或文本与音频输入,同一指数得 15.95。

    ly. d1-3B scores 48.57 on the Decision Index v0.2.1 (public split), ahead of every model under 10B and on par with Decider 35B-A3B, a decision model 12x its size. It runs the full NVIDIA stack, from DGX in the data center to Jetson at the edge: d1-3B answers a question in 8 ms on an NVIDIA GeForce RTX 4090, 16 ms on a Jetson AGX Thor, and 26 ms on a Jetson AGX Orin. Even the Jetson Orin Nano runs it in 50 ms, fast enough for real-time decisions on the smallest edge hardware. d1-omni-600M is our

    匹配字段正文

星期二

  1. Latent Space精选

    Reflection 发布 Beam:501B 总参数、23B 激活的美国开源 MoE 模型

    Reflection 发布 Beam,一个面向编码、智能体与科研的纯文本 MoE 模型,总参数 501B、激活 23B,从零训练,Apache 2.0 全量权重预计本月发布。团队称预训练用了 23.8T token,并在约 10,500 块 GB300 上各跑四周预训练与 RL,自报 SWE-bench Verified 得分 80.9。观察者认为 Beam 大致处于 GLM-5.2 水平,部分基准低于 DeepSeek V4 Flash。

    unning at once (sandboxes); Positioning: Observers place Beam around GLM-5.2 level (iScienceLuvr) and below DSv4 Flash on some benchmarks (critique). Nathan Lambert groups it with Nvidia and Thinking Machines as strong US releases that still trail Chinese counterparts (Lambert); Other open and specialized models:Aleph Alpha Kolibri: 78B total / 3.46B active, Apache 2.0, built for German and English. Self-reported scores are 96.9% AIME 2025, 84.3% GPQA Diamond and 66.4% SWE-Bench Verified (summar

    匹配字段正文

  2. TechCrunch AI

    Reflection 发布开源权重模型 Beam,对标中国开源模型并压低算力成本

    Reflection AI 正式发布其首个前沿开源权重模型 Beam,称在高级推理基准上与中国领先开源模型持平,且推理算力消耗少 3-4 倍。Beam 为文本 MoE 模型,总参数 5010 亿、激活 230 亿,预训练 23.8 万亿 token,上下文窗口 100 万 token;对比 Z.ai 的 GLM-5.2 总参数约 7440 亿、激活 400 亿。公司称本月将公开权重与完整技术细节,并通过超大规模云厂商和 neocloud 分发,性能主张尚未经独立验证。

    g is a multimodal model and Beam is text-only. Reflection was founded in 2024 by two former Google DeepMind researchers and has raised roughly $4.7 billion from backers, including Nvidia, Sequoia Capital, and Lightspeed Venture Partners, per PitchBook. Its last round valued the company at a $25 billion pre-money valuation . The startup has also been locking up compute — a key ingredient needed to train frontier models capable of luring customers away from Anthropic’s and OpenAI’s closed models,

    匹配字段正文

星期日

  1. MarkTechPost

    Aleph Alpha 发布 Kolibri:78.1B 开源英德双语 MoE 模型,每 token 仅激活 3.46B 参数

    Aleph Alpha 发布开源权重英德双语 MoE 模型 Kolibri,总参数 78.1B,每 token 仅激活 3.46B(4.4%),支持最长 1,048,576 token 上下文,并以 Apache 2.0 许可在 Hugging Face 发布。FP8 权重约 78GB,可在单张 B200、B300 或 H200,或 2 张 H100 SXM5 上通过 vLLM 部署,面向公共管理、工业和航空航天等受监管行业的自主部署。模型采用 50 层 Transformer、384 个路由专家中每 token 选 6 个加 1 个共享专家,40 层滑窗注意力配 10 层全注意力,预训练覆盖 20T token。

    GPT-5 tokenizer. That means 11.2% fewer tokens on German web text. In English, Kolibri reaches 4.58 bytes per token against 4.67 for GPT-5. Pre-training covered 20T tokens on 768 NVIDIA B200 GPUs, followed by 3.44T mid-training tokens at 65,536 sequence length. A 201B-token long-context stage then trained on 262,144-token sequences. Aleph Alpha added more than 2T German tokens it curated from the web or generated synthetically. Post-training combined supervised fine-tuning, mixed with MergeMix,

    匹配字段正文

星期四

  1. The Rundown AI

    Google 发布 Gemini 4 Argon,19 项基准中 13 项超过 GPT-6 Astra 与 Claude Opus 5.5

    Google 发布新前沿模型 Gemini 4 Argon,在 Google 自测的 19 项基准中有 13 项超过 GPT-6 Astra 和 Claude Opus 5.5,目前仅向经过审核的特定网络安全团队开放。该模型在 Arena 文本榜登顶,AA 智能指数得 53 分,仅次于 Opus 5.5,与 Fable 5.1 和 Astra 持平,并在真实编码测试 DeepSWE 上取得 77.9% 的领先成绩。Google 尚未公布更大范围上线时间,API 定价为每百万 token 输入 2 美元、输出 10 美元,促销期结束后升至 4 美元和 20 美元;彭博社报道称内部对 Argon 的编码能力存疑,Google 否认了这一说法。

    ts Season 3, featuring 31 days of hands-on tutorials on building and securing AI agents in production with Gemini and ADK — no sign-in or paywall. OpenAI, Google, Anthropic, Meta, Nvidia, and xAI signed a voluntary White House “Superintelligence Accord” to police their top models with outside auditors. The Pentagon’s Pete Hegseth introduced Project Meridian, a 120-day hunt for futuristic weapons troops could need “from under the Earth to beyond the moon,” with Elon Musk, Newt Gingrich, and Palme

    匹配字段正文

  2. Latent Space精选

    Google DeepMind 发布 Gemini 4 Argon,输出上限提至 1M token

    Google DeepMind 发布前沿模型 Gemini 4 Argon,面向编码、企业知识工作和网络安全防御,今日起通过 Fairwind Program 向受信任测试者开放,并承诺尽快扩大访问。Google 称其输出上限达 1M token,较此前的 64K 大幅提升,标准定价为每 1M 输入/输出 token 4 美元/20 美元,另有 50% 的限时折扣。Google 称 Argon 在 19 项公开基准中的 13 项排名第一,DeepSWE 得分 77.9%,高于 Opus 5.5 的 74.2% 和 Astra 的 74.1%;Artificial Analysis 测得其在 Intelligence Index 上得 53 分,与 GPT-6 Astra 持平。

    bug fix : OpenAI fixed an image-encoding bug, adding 1 Intelligence Index point to GPT-6 Luna. Ultrafast inference : OpenAI quotes up to 300 tok/s. SemiAnalysis reports it runs on NVIDIA GPUs at low batch sizes, not on Cerebras ( @kimmonismus ). Hands-on report : Generation is about 8x faster, but end-to-end agent tasks speed up only 2–4x because tool latency dominates ( @sayashk ). Computer use : Gains are largest here, since UI actions respond in milliseconds. Cost : The tester exhausted a wee

    匹配字段正文

星期二

  1. The Rundown AI

    Anthropic 发布 Claude Sonnet 5.5,速度提升 30% 且价格减半

    Anthropic 推出 Claude Sonnet 5.5,这是加入 5.5 家族的中端模型,速度提升 30%,在知识工作和编码上较上一代 Sonnet 有明显提升,部分测试接近 Opus 5.5 而价格仅为其一半。该模型在 AA 智能指数上得分为 56,仅次于 Opus 5.5,超过 Fable 5.1 和 GPT-6 Astra 的 53;在多项办公测试中几乎追平 Opus 5.5,编码基准上接近或领先 Opus 与 Astra。Anthropic 表示部分任务成本最多降低 30%,并首次为 Sonnet 层级模型加入网络安全护栏。

    . The details: Li wrote that AMD CEO Lisa Su backed the lab early on, and the teams have been tuning how World Labs' models train and run on AMD GPUs since last year. Both AMD and Nvidia invested in World Labs' $1B round in February, with the deal now giving AMD a lab that its rival (and the CEO’s cousin) was also funding. Li will report directly to Su, saying that her team needs to get “closer to the hardware” since AI without that focus stays “hobbled in efficiency.” World's first product, Mar

    匹配字段正文

星期一

  1. Hugging Face

    H 公司发布 Holo4 通用计算机操作智能体模型

    H 公司发布 Holo4 系列通用计算机操作智能体模型,可在桌面、网页、Android、代码沙箱和业务 API 上以同一模型同一调用方式运行,并能点击输入、编写执行代码以及调用 MCP 或 API 工具。在 OSWorld 2.0 上,Holo4 27B 得分 61.7%,Holo4 35B-A3B 为 30.9%,官方称其以远少得多的参数量和更低成本接近前沿闭源模型。两个尺寸已在 H Models API 上线,权重以 BF16、FP8、NVFP4 和 4-bit GGUF 格式发布在 Hugging Face,公开基准的全部轨迹也已开源。

    ary. Holotron4 Nano Our post-training stack is designed to adapt to new foundation models and produce agents that generalize across interfaces and environments. As a member of the NVIDIA Nemotron Coalition , we applied our latest stack to the Nemotron 3 Nano Omni model as a follow-up to Holotron 3. The same recipe turns Nemotron 3 Nano Omni into Holotron4 Nano, a generalist agentic model that significantly improves over the base model on GUI workflows and in environments exposing MCP, APIs or co

    匹配字段正文

星期三

  1. The Rundown AI

    Anthropic 与 OpenAI 相隔 90 分钟发布 Claude Opus 5.5 和 GPT-6 Sol、Luna

    Anthropic 发布 Claude Opus 5.5,在 AA 的 Intelligence Index 上以 58 分登顶,超过此前领先的 Fable 5.1 和 GPT-6 Astra 的 53 分,价格比前代低 40%;OpenAI 随后推出 GPT-6 Sol 和 Luna,得分较 5.6 对应版本略有提升,价格减半,Luna 为每百万 token 0.10/0.50 美元,Sol 为 2/10 美元。Anthropic 称 5.5 在其内部对齐基准上取得迄今最好成绩,并改进了被批评的 Claudish 写作风格;这是 Anthropic 提出 pacing 后的首个发布,Sam Altman 表示 pacing 并不意味着停止。

    superintelligence. This conference gives you access to: 3 tracks and 30+ technical breakout sessions 4+ hands-on keyboard labs led by CoreWeave engineers Keynotes from CoreWeave, NVIDIA, Capital One, and more Entertainment from BattleBots, Pitbull, and F1 Image source: Images 2.5 / The Rundown The Rundown: OpenAI says a single internal model has resolved 100+ open math problems since it began training on Aug. 28, and it’s now handing nine top mathematicians an advisory role in how those results

    匹配字段正文

  2. Latent Space

    Anthropic 发布 Claude Opus 5.5,OpenAI 同日推出 GPT-6 Sol 与 Luna

    Anthropic 发布 Claude Opus 5.5,称其为新 Claude 5.5 家族首个模型,多数任务达到 Claude Fable 5.1 水平,运行成本比 Opus 5 低 40%,并已成为 Claude Code 和 Claude 应用的默认模型。OpenAI 约一小时后发布 GPT-6 Sol 和 Luna,定价分别为每百万 token 输入/输出 2 美元/10 美元和 0.10 美元/0.50 美元,均比 GPT-5.6 前代便宜约 50%。Opus 5.5 的系统卡还报告了最多 100 个并行智能体的多智能体扩展结果,Sonnet 5.5 和 Haiku 5.5 将在未来几周跟进。

    nd individuals trying to run large open-weight models with constrained memory, including growing KV caches. Hardware and local deployment were another theme. A hands-on post about NVIDIA DGX Spark described a 15×15×5.05 cm , 1.2 kg system with GB10 Grace Blackwell , 128 GB unified memory , and up to 1 PFLOP FP4 sparse theoretical compute , with NVIDIA claiming support for local inference on models up to 200B parameters with quantization and fine-tuning up to 70B with methods like QLoRA @kimmonis

    匹配字段正文

星期二

  1. Latent Space

    小米发布 MiMo-V2.6-Pro 与 Flash 开源全模态模型,训练成本约 300 万美元

    小米发布 MiMo-V2.6 系列开源全模态模型,其中 MiMo-V2.6-Pro 为 1.02T 总参数、42B 激活参数,采用 MIT 许可,Artificial Analysis 称其以 46 分登顶开源权重模型智能指数。技术报告显示其 RL 训练在 130 小时、75B token 上花费约 260 万美元,并计划开源环境代码与训练配方,但 7k+ 任务数据集尚未发布。

    xed-precision quantization averaging 2.38 bits/weight , including custom CUDA kernels in patched llama.cpp. On the edge/local side, @vikhyatk released Parakeet Redux , compressing NVIDIA’s speech model from 1.2GB to 178MB , running at 113x realtime on CPU , while beating the base model on 25-language FLEURS and staying within 0.3 WER on English. Agents, Security, and Human-in-the-Loop Control Computer-use systems are becoming more productionized, but security is now central : Patrick Wardle repo

    匹配字段正文

  2. Black Forest Labs Blog精选

    Black Forest Labs 发布 FLUX 3 Action:7B 开源世界动作模型用于机器人控制

    Black Forest Labs 发布 FLUX 3 Action,一个从多模态 FLUX 3 主干衍生的 7B 开源权重世界动作模型,在 RoboLab-120 榜单上以不到此前最佳开源模型一半的参数量取得 38.3% ± 0.38 的成功率,速度最高提升 3.95 倍。该模型在图像、视频和音频数据上预训练,其中视频占训练 token 的 95% 以上,再通过联合视频-动作训练和针对目标本体的微调适配动作空间。报告还分析了将其与 GPT 6 Astra 等推理模型组合的混合方案,发现快速控制能让具身推理更省时省钱,每美元成功率最高提升 53.64%。

    osmos 3 Nano in FP8 across consumer, workstation and datacenter GPUs. 1Yang et al. 2026, RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies. ↩ 2NVIDIA et al. 2026, Cosmos 3: Omnimodal World Models for Physical AI. ↩ 3Intelligence et al. 2025, π0.5: A Vision-Language-Action Model with Open-World Generalization. ↩ π0.5 was evaluated only in BF16; both precision views show its BF16 results. FLUX 3 Action; Cosmos 3 Nano; π0.5; Previous Pareto front; New Pareto fro

    匹配字段正文

星期六

  1. A6 a6-047 · Baseten 工程博客精选

    DeepSeek 发布 V4.1-Flash:面向编码智能体的高效 prefill 模型

    DeepSeek 本周发布并开源 DeepSeek-V4.1-Flash,这是一个 552B 参数、8B 激活参数用于 prefill、16B 用于 decode 的多模态 MoE 模型,支持 1M token 上下文窗口,已上线 HuggingFace 与 Baseten Model APIs。它采用 Causal Encoder-Decoder 架构,prefill 只跑 20 层编码器,decode 跑完整模型,官方称全局 KV cache 内存仅为 V4-Flash 的四分之一。在编码与智能体基准上,V4.1-Flash 以约三分之一的总参数量超过 V4-Pro,长程智能体任务提升达两位数,但 Automation-Bench 得分 54.8,复杂工作流仍有约一半失败。

    DeepSeek-V4.1-Flash

    Capturing that reuse in production depends on how requests are routed across GPUs, which is where the serving stack comes in. The Baseten Inference Stack handles that routing with NVIDIA Dynamo and KV cache-aware routing, which steer each request to the replica already holding its prefix instead of whichever replica is free. Taken from 2x faster inference with KV cache-aware routing All of this runs behind Baseten Model APIs , where DeepSeek-V4.1-Flash is available now. Try it in our Model Libra

    匹配字段正文

星期五

  1. The Rundown AI

    OpenAI 发布 GPT-6 Astra,称其为最智能且最对齐的模型

    OpenAI 发布 GPT-6 Astra,称其为全球最智能且最对齐的模型,在科学、数学、计算机使用、编码、网络安全等领域的基准测试创下新高。Astra 在 ARC-AGI-3 上达到 99.9%,此前 GPT-5.6 Sol 为 7.8%,FrontierMath T4 为 98%,ExploitBench 网络安全为 100%,但在 AA 智能指数上得 61 分,落后于 Fable 5.1、5、Opus 5 和 Meta 的 Muse Spark 1.3。API 定价为每百万 token 10 美元和 50 美元,约为 GPT-5.6 Sol 的 2.5 倍,但单任务 token 使用更高效;先向少数机构开放,付费 ChatGPT 套餐和 API 将在数日内跟进,并提供 Astra Pro 档位。

    lagship, top-rated model 🗣️ Realtime-TTS-2 - Inworld's top-ranked voice model, now generally available 🌎 Atlas - World Labs’ new SOTA world model in early access Sponsored Listing Nvidia announced the acquisition of Hugging Face for $12.9B, with CEO Jensen Huang saying the platform will remain open to every cloud and chip. Axiom Math published a paper breaking a decade-old record by proving prime numbers never stop appearing within 212 of each other, only for OpenAI's GPT-6 Astra to shrink that

    匹配字段正文

星期五

  1. 蚂蚁百灵:Developer Blog精选

    蚂蚁百灵发布金融增强模型 Ling-3.0-flash-Fin

    蚂蚁百灵发布首个金融增强模型 Ling-3.0-flash-Fin,延续 Ling-3.0-flash 架构,保持 124B 总参数、5.1B 激活参数,通过金融语料持续预训练、领域后训练和工具使用优化强化年报、财务工作簿等复杂金融内容处理。模型在 FinFIRST、FinSearchComp Verified、Finance Agent、APEX-Agents、SpreadsheetBench、τ³-Banking 等基准上测试,AA Intelligence Index v4.1.1 得分由 38 提升至 41。模型将在 OpenRouter 开启为期一个月的限时免费 API 调用,权重于下周开源。

    图

    或最终产品界面;演示内容仅供参考,不构成任何投资建议或决策依据 在以下两个案例中,Ling-3.0-flash-Fin 将分散材料整理为“事实—解释—判断—图表”的证据链: Micron:还原合同信息的经济含义。 模型将战略客户协议及约 1000 亿美元剩余履约义务放回多年合同语境,区分未来履约约束与当期收入,并保留合同执行、价格调整和供需变化等判断边界。 NVIDIA:从新披露口径识别边际增长来源。 Hyperscale 与 ACIE 业务规模接近,环比增速分别为 12% 和 31%。模型据此提炼出,边际增长动力可能向 AI 云、主权 AI、企业和工业客户扩散,并通过并列图表将研究判断与官方披露逐一对应。 在现有案例中,Ling-3.0-flash-Fin 能够围绕核心命题组织数据、分析文字和图表,让每项证据服务于研究结论。在视觉表达上,模型根据内容关系选择图表形式、建立信息层级,生成便于后续编辑、路演沟通和专业审核的研究材料。 06 一个完整的案例:四项能力如何汇入同一条任务链 为了观察这些能力如何协同,我们选取了一个源自 APEX-Agents 投资银行任务场景的 Planet

    匹配字段正文

星期四

  1. The Rundown AI精选

    Z AI 确认神秘模型 Ox Alpha 即 GLM-5.3-Flash

    Z AI 确认匿名免费模型 Ox Alpha 是 GLM-5.3-Flash,并公开模型权重。该模型登顶 OpenRouter,使用量为第二名 DeepSeek 的两倍;Z AI 称免费开放期间的服务完全运行于中国制造的芯片,Artificial Analysis 给出的单任务价格为 $0.045,较同档模型低 10 倍。

    epSeek, a debut the platform ranked as its biggest ever. Z AI said the free usage week ran entirely on Chinese-made chips, claiming the setup now serves tokens about as cheaply as Nvidia hardware. Artificial Analysis grades the model at a 57 on its Intelligence Index for a discounted $0.045 per task, 10x lower than similarly-ranked rivals. Why it matters: The mystery that captivated the AI world for a week is over, and it was the original suspect all along (despite some odd posturing from Google

    匹配字段正文

星期二

  1. Hugging Face精选

    IBM 发布 Granite 4.2 语言模型,详解其构建流程

    IBM 发布 Granite 4.2 系列语言模型,包含 3B、8B、30B 三个尺寸,均可输出思维链并支持 thinking 与非 thinking 模式,另设低推理预算的 low-effort 模式。8B 和 30B 额外经过智能体 RL 阶段,学习调用工具、编辑运行代码、操作终端和联网搜索,全部模型支持原生工具调用,可通过 OpenAI 兼容接口接入智能体框架。模型基于 Granite-4.1 base 后训练,SFT 数据约 720 万条、约 100B token,RL 采用异步 GRPO 分阶段课程,并已发布 FP8、NVFP4、MXFP4 和 GGUF 量化版本。

    ical llama.cpp tool as described in https://github.com/IBM/gguf gguf-conversion--quantization . Several GGUF formats are provided: We trained the Granite 4.2 language models on an NVIDIA GB200 NVL72 cluster hosted by CoreWeave, featuring: A 72-GPU NVLink domain for high-speed intra-rack communication A non-blocking Fat-Tree NDR 400 Gb/s InfiniBand fabric for full-bandwidth inter-rack connectivity Thousands of GPUs operating at cluster scale This infrastructure delivers the high-bandwidth, low-la

    匹配字段正文

星期一

  1. The Rundown AI

    AI 前沿的神秘挑战者:Ox Alpha

    匿名模型 Ox Alpha 免费登陆 OpenRouter,支持 1M-token 上下文和多模态输入,面向编程与持续智能体任务。其 DeepSWE 子集完整测试得分为 63%,接近 Fable 5 且每项任务消耗的 Token 更少;早期线索指向智谱 AI,但开发者身份尚未揭晓。

    ts by up to 36% . Hugging Face , which hosts over 2M models and 1.5M datasets, is exploring a sale at $13B or more, up from its $4.5B valuation in 2023, Business Insider reported. Nvidia struck a $6B deal to license Poolside’s model-development technology and bring 100+ engineers onto the Nemotron team to build powerful open-weight models. Dr. Dre told the NYT he’s using AI in producing music, calling it a tool for creativity and noting that the only people who see it as a threat are those who h

    匹配字段正文

星期四

  1. Liquid AI 模型与工程博客

    Liquid AI 发布 LFM2.5-DSpark 草稿模型,推理最高提速 3.2 倍

    Liquid AI 为 LFM2.5-1.2B-Instruct、LFM2.5-2.6B 和 LFM2.5-8B-A1B 发布 DSpark 投机解码草稿模型,GPU 上吞吐最高提升 3.18 倍,端侧最高 2.87 倍,且不改变输出质量。草稿模型约 300M 参数,采用 5 层、block size 9 的简化注意力结构,已在 Hugging Face 提供 Safetensors 与 GGUF 权重,并开源集成到 llama.cpp 和 SGLang。在 BFCL 多工具函数调用场景中,LFM2.5-2.6B 的延迟平均降低 57%;LFM2.5-8B-A1B 受 llama.cpp Metal 后端 MoE 实现影响,端侧仅提升 18%。

    is traditionally memory-bound. Most of the latency comes from streaming weights from DRAM into SRAM, not from intense computation. This is true for both powerful GPUs, such as the NVIDIA H100, and for edge devices like MacBooks or iPhones. Speculative decoding addresses this issue by using a lightweight draft model to produce candidate tokens. The target model then verifies them all in a single forward pass, sharing the cost of loading the weights across all tokens we are verifying. Over the yea

    匹配字段正文