←搜索结果

按匹配度 · 共 11 条结果 清除搜索

星期六

  1. MarkTechPost

    决策 AI 模型解析:TypeSafe Jev 与 Fastino GLiDE、GLiNER2.5-Decide 及开源竞品对比

    决策 AI 模型是一类返回选择、分数或概率而非生成文本的模型,TypeSafe 在隐身两年后推出 Jev,Fastino Labs 三周内跟进两款竞品。Jev 支持 Choice、Score、Noul 三种原语,定价为每百万输入 token 0.042 美元、输出免费,TypeSafe 称端到端响应为 70 至 500 毫秒。在 TypeSafe 的 4 项工作流评测中,Jev 以 67.8% 平均准确率、每例 0.4 秒追平 Claude Sonnet 5,但落后最强前沿配置 6.3 个百分点。

    s. Langfuse labels the feature “decision-model evaluators” so other models can be added later. Its Jev support is still marked experimental. 4. 6G edge network orchestration A new arXiv paper used Jev to interpret service contracts at the network edge. At matched correctness, Jev cut median decision latency by 22.4% versus DeepSeek and 61.9% versus Gemini. 5. Real-time game agents TypeSafe’s Doom demo ran Jev at 10 queries per second. The team estimated the cost at roughly $7 per hour. Competito

    匹配字段正文

星期五

  1. LMSYS Blog精选

    LMSYS 公开 infer-forge:围绕 SGLang 的 Harness、Loop 与 Graph 工程方法

    LMSYS 发布 infer-forge 方法论文,介绍围绕 SGLang 的推理工程系统,由 MonoRepo、Harness、Task Loop 和 Task Graph 四层结构组成,用于让跨仓库的推理优化工作可复现、可验证。该系统为内部工程系统,并非 SGLang 或 LMSYS 官方组件,目前不开源,文章公开的是构建方法论,供团队交给 AI 编码工具并结合自身系统上下文搭建适配实现。文中给出使用记录:一位工程师 4 月至 7 月的在飞任务峰值从 2 个升至 9 个,DeepSeek-V4-Pro 服务项目中 38 个可独立验证的任务节点被组织成任务图。

    The inference deployment space stacks five layers of static configuration. Model shows Ling, Qwen, DeepSeek, Kimi, GLM, and MiniMax. Serving Scenario runs Modality (text, image, video) into Traffic Shape (input and output length, media count, resolution, QPS, concurrency, cache reuse) into SLO (TTFT, TPOT, throughput, E2E latency). Serving Topology separates Deployment Architecture—Colocated PD, PD Disaggregation, and EPD Disaggregation, each listing the node roles it is built from — Prefill and Decode together, then Prefill and Decode as separate roles, then Encoder alongside them — from Parallelism—TP, PP, DP, and EP—because the two are chosen independently. Versioned Runtime Profiles is a stack of tabbed cards labelled Service A rev. 12, Service B rev. 7, and Service C rev. 21, plus a fourth paler card behind them all, blank and showing only its top edge, whose narrow tab carries an ellipsis for the profiles not drawn, the front card holding an Engine Configuration and a Container I

    ploration, LMSYS Org, July 2, 2026; Lilian Weng — Harness Engineering for Self-Improvement, Lil'Log, July 4, 2026; Boye Niu et al. — Flow: Modularized Agentic Workflow Automation, arXiv:2501.07834, 2025; Andy Xu and Yu-Wing Tai — Meta-Agent: From Task Descriptions to Verified Multi-Agent Systems, arXiv:2605.25233, 2026; Ao Li et al. — GraphFlow: A Graph-Based Workflow Management for Efficient LLM-Agent Serving, arXiv:2605.22566, 2026; Sydney Runkle and Harrison Chase — 3 Years of Graph Engineeri

    匹配字段正文

星期六

  1. Ahead of AI

    如何控制大语言模型的推理力度

    文章解释了大语言模型如何通过不同推理力度,在回答质量、token 消耗、成本和延迟之间进行权衡。其实现通常结合混合 SFT 数据、按模式调整的 RL 奖励、长度惩罚、独立专家模型或硬性 token 预算;DeepSeek V4、Nemotron 3 Ultra、Kimi K3、GLM-5、Qwen3 和 Inkling 采用了不同组合。模型规模与推理力度是两套独立控制方式,较小模型提高推理力度有时可达到与较大模型降低推理力度相近的基准成绩。

    d excitement around reinforcement learning with verifiable rewards and the development of reasoning models, there is another paper, Kimi K1.5, published on exactly the same day on arXiv (22 Jan 2025). Also, the term RLVR was already coined two months earlier in Tülu 3: Pushing Frontiers in Open Language Model Post-Training. One reason why the DeepSeek R1 is ultimately the more popular paper is that it demonstrated that reasoning behavior can be achieved with pure reinforcement learning (RL). For

    匹配字段正文

星期六

  1. Ahead of AI

    LLM 研究论文清单:2026 年 1 月至 5 月

    作者整理了 2026 年 1 月至 5 月收藏的 LLM 研究论文,并明确表示这不是完整收录,而是基于个人工作兴趣筛选的参考清单。内容侧重推理模型、强化学习和高效推理,同时涵盖 Agent 框架、工具使用、长上下文、扩散语言模型及服务基础设施。

    BF16, synthetic MMLU-style data, and post-training quantization recipes, but covering these in detail would be out of scope for this overview. 1 Jan, Deep Delta Learning, https://arxiv.org/abs/2601.00417; 6 Jan, MiMo-V2-Flash Technical Report, https://arxiv.org/abs/2601.02780; 13 Jan, Ministral 3, https://arxiv.org/abs/2601.08584; 29 Jan, Scaling Embeddings Outperforms Scaling Experts in Language Models, https://arxiv.org/abs/2601.21204; 30 Jan, LatentLens: Revealing Highly Interpretable Visual

    匹配字段正文

星期三

  1. Ahead of AI

    2026 年 1—2 月开放权重 LLM 架构对比:10 组主要模型发布

    作者对 2026 年 1—2 月的主要开放权重 LLM 发布进行架构对比,并补充了 3 月 6 日发布的 Sarvam 30B 和 105B。文章逐一分析 Trinity Large、Kimi K2.5、Step 3.5 Flash、Qwen3-Coder-Next、GLM-5、MiniMax M2.5、Nanbeige 4.1 3B、Qwen3.5、Ling 2.5、Ring 2.5 与 Tiny Aya,重点比较 MoE、混合注意力、MLA、Gated DeltaNet、滑动窗口注意力和 MTP 等设计。作者认为,近期架构普遍加入计算效率优化,但模型表现也很大程度取决于数据质量和训练方案。

    3B active parameters) and Trinity Nano (6B with 1B active parameters) Along with the model weights, Arcee AI also released a nice technical report on GitHub (as of Feb 18 also on arxiv) with lots of details. So, let’s take a closer look at the 400B flagship model. Figure 2 below compares it to z.AI’s GLM-4.5, which is perhaps the most similar model due to its size with 355B parameters. As we can see in the Trinity and GLM-4.5 comparison, there are several interesting architectural components ad

    匹配字段正文

星期五

  1. vLLM 官方博客

    深入 vLLM:高吞吐 LLM 推理系统架构解析

    文章系统解析 vLLM V1 如何从单 GPU 离线推理扩展到多 GPU、多节点异步服务。内容涵盖调度器、PagedAttention、连续批处理、前缀缓存、引导解码、推测解码、P/D 分离及分布式请求链路,并说明如何用 vllm bench 衡量延迟与吞吐量。分析基于 2025 年 8 月 9 日的 commit 42172ad。

    Ashish Vaswani for reading pre-release version of this blog post and providing feedback! References vLLM https://github.com/vllm-project/vllm; "Attention Is All You Need" https://arxiv.org/abs/1706.03762; "Efficient Memory Management for Large Language Model Serving with PagedAttention" https://arxiv.org/abs/2309.06180; "DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model" https://arxiv.org/abs/2405.04434; "Jenga: Effective Memory Management for Serving LLM with H

    匹配字段正文

星期一

  1. Mario Zechner 原创博客精选

    把提示词当代码:用 port.md 和 JSON 状态做跨语言代码移植

    作者提出把 LLM 视为一台缓慢且不可靠的计算机,用自然语言编写程序:提示词是程序,JSON 和 Markdown 文件是持久状态,工具调用是 I/O。他以 Spine 运行时从 Java 到 C++ 的移植为例,用 port.md 定义工作流、用 porting-plan.json 记录每个类型的 portingState 以便随时恢复,并借助 jq 精确读写状态、通过 MCP 扩展在 VS Code 中保留人工检查点。原本需要 2-3 周的移植工作现在缩短到 2-3 天,但复杂类型层级和执行流仍需人工判断。

    code" in the form of prompts that execute on the LLM to produce the actual code that runs on real CPUs. Yes, I know LLMs aren't actually computers (though there are some papers on arXiv...). The metaphor is a bit stretched. But here's the thing: as developers, we're used to encoding specifications in precise programming languages. When we interact with LLMs, the fuzziness of natural language makes us forget we can apply the same structured thinking. This framework bridges that gap: think "inputs

    匹配字段正文

星期二

  1. vLLM 官方博客

    vLLM 结构化解码入门与 XGrammar 集成解析

    vLLM 介绍结构化解码的原理与实现,并已支持 Outlines 和 XGrammar 两种后端,以约束大语言模型输出符合指定结构。XGrammar 集成在负载下可将每个输出 token 的生成时间改善至多 5x,不适用的请求会回退至 Outlines。规划中的 v1 将重点优化性能,并探索调度器级掩码广播以支持混合请求批处理。

    top level view of structure decoding

    inuous effort to improve structured decoding in vLLM. References Bahdanau, D., Cho, K., & Bengio, Y. (2016). Neural Machine Translation by Jointly Learning to Align and Translate. arXiv preprint arXiv:1409.0473; Haugeland, J. (1997). Mind Design II: Philosophy, Psychology, and Artificial Intelligence. The MIT Press. https://doi.org/10.7551/mitpress/4626.001.0001; Hendler, J. (2008). Avoiding Another AI Winter. IEEE Intelligent Systems, 23(2), 2–4. https://doi.org/10.1109/MIS.2008.20; Hochreiter,

    匹配字段正文

星期四

  1. Anthropic Engineering精选

    Anthropic 推出 Contextual Retrieval,将 RAG 检索失败率降低 49%

    Anthropic 提出 Contextual Retrieval 方法,通过 Contextual Embeddings 和 Contextual BM25 两项子技术,将 top-20 检索失败率降低 49%(5.7% 降至 2.9%),叠加 reranking 后降低 67%(5.7% 降至 1.9%)。该方法用 Claude 3 Haiku 为每个分块生成 50-100 token 的上下文说明,再拼接到分块前进行嵌入和建立 BM25 索引,配合 prompt caching 时生成上下文化分块的一次性成本为每百万文档 token 1.02 美元。Anthropic 同时提供了可直接部署的 cookbook。

    chunk, the one-time cost to generate contextualized chunks is $1.02 per million document tokens . Methodology We experimented across various knowledge domains (codebases, fiction, ArXiv papers, Science Papers), embedding models, retrieval strategies, and evaluation metrics. We’ve included a few examples of the questions and answers we used for each domain in Appendix II . The graphs below show the average performance across all knowledge domains with the top-performing embedding configuration (G

    匹配字段正文

星期五

  1. Lilian Weng:Lil'Log

    LLM 驱动的自主智能体:规划、记忆与工具调用综述

    Lilian Weng 在 Lil'Log 撰文梳理以 LLM 为核心控制器的自主智能体架构,将其拆解为规划、记忆和工具调用三大组件。规划部分涵盖 CoT、Tree of Thoughts、LLM+P 等任务分解方法,以及 ReAct、Reflexion、Chain of Hindsight、Algorithm Distillation 等自我反思机制。记忆部分区分短期上下文学习与基于外部向量库的长期记忆,并介绍 LSH、ANNOY、HNSW、FAISS、ScaNN 等快速 MIPS 检索算法;工具调用部分则列举 MRKL、TALM、Toolformer、HuggingGPT、API-Bank 以及 ChemCrow 等科学发现智能体案例。文末指出有限上下文长度、长期规划与任务分解、自然语言接口可靠性三类常见局限。

    al. “Chain of thought prompting elicits reasoning in large language models.” NeurIPS 2022 [2] Yao et al. “Tree of Thoughts: Dliberate Problem Solving with Large Language Models.” arXiv preprint arXiv:2305.10601 (2023). [3] Liu et al. “Chain of Hindsight Aligns Language Models with Feedback “ arXiv preprint arXiv:2302.02676 (2023). [4] Liu et al. “LLM+P: Empowering Large Language Models with Optimal Planning Proficiency” arXiv preprint arXiv:2304.11477 (2023). [5] Yao et al. “ReAct: Synergizing

    匹配字段正文

星期三

  1. Lilian Weng:Lil'Log

    Lilian Weng 长文梳理提示词工程方法:从零样本到 CoT 与工具调用

    Lilian Weng 发布提示词工程综述,系统梳理零样本、少样本、指令提示、CoT、自洽采样、检索增强与外部工具调用等方法,并指出提示词效果因模型而异,需要大量实验和启发式调整。文章还汇总了示例选择与排序、自动提示词生成、Tree of Thoughts、Toolformer 等方向的研究,并附 26 篇论文引用。

    emantic Kernel [1] Zhao et al. “Calibrate Before Use: Improving Few-shot Performance of Language Models.” ICML 2021 [2] Liu et al. “What Makes Good In-Context Examples for GPT-3?” arXiv preprint arXiv:2101.06804 (2021). [3] Lu et al. “Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity.” ACL 2022 [4] Ye et al. “In-Context Instruction Learning.” arXiv preprint arXiv:2302.14691 (2023). [5] Su et al. “Selective annotation makes language models better f

    匹配字段正文