←搜索结果

按匹配度 · 共 51 条结果 清除搜索

星期二

  1. Modal 官方工程博客

    Modal 如何实现真正的无服务器 GPU

    Modal 官方工程博客详解其无服务器 GPU 方案,通过云缓冲、自定义文件系统 ImageFS、CPU 检查点恢复和 CUDA 检查点恢复四项优化,把 AI 推理服务副本的启动时间从约 2000 秒缩短到 50 秒,提速 40 倍。其中 GPU 内存快照让 vLLM 或 SGLang 服务约 1 GiB 模型(Qwen 3 0.6B bf16)的冷启动典型提速 4-10 倍,Reducto 的冷启动从约 70 秒降到约 12 秒。作者称该技术栈已在数千万个副本规模上运行,过去三个月有数百家组织使用 CPU 和 GPU 快照。

    Diagram indicating latency to scale up an inference server in a baseline cloud system and on Modal

    raw algorithmic operation requirements by aggregate arithmetic bandwidth. This is catnip for engineers. It’s also especially critical for “hero run” large-scale training, so it draws a lot of investment and attention, e.g. recently as everyone dunked on xAI’s 10% MFU. But at the other end of the stack, there’s a more basic form of utilization that wrecks the relationship between achieved output and allocated capacity for inference workloads, GPU Allocation Utilization: GPU Allocation Utilizatio

    匹配字段正文

星期六

  1. Browser Use

    Browser Use 开源库的生产架构:SQS 到 Lambda 的工程实践

    Browser Use 官方博客公开了其开源库在生产环境运行的架构,采用 FastAPI on ECS Fargate 接收任务、单条标准 SQS 队列传递消息、Lambda 作为 worker 执行 Agent,并将检查点、截图、日志和输出文件存入 S3。由于 Lambda 有 15 分钟硬限制,系统在截止前两分钟优雅停止、将状态写入 S3 并重新入队,由新的 Lambda 调用恢复继续执行,continuation 计数器上限为 12,约合三小时墙钟时间。失败处理直接复用 Lambda 重试与 SQS 死信队列,三次失败后进入 DLQ 且不自动重投;作者还提到 Lambda 的 /tmp 在热实例间不会自动清理,因此每次调用开始时清空工作区。

    run carrying the task ID and execution config. No ordering, no deduplication, no separate queue per workload type since agent tasks are independent of each other. The worker is an AWS Lambda function with from browser use import Agent at the top. It pulls messages off SQS, instantiates the agent, runs it to completion, and writes results to S3. The complexity is in how each piece handles failure. API entry The handler validates the payload, creates session and task rows in the database, commits,

    匹配字段正文

星期二

  1. METR Notes

    METR 用 Opus 4.6 复刻《杀戮尖塔》与《小丑牌》CLI 版的观察

    METR 用简单 ReAct 脚手架让 Opus 4.6 复刻《杀戮尖塔》和《小丑牌》的 CLI 版本,给出 6000 万 token 总量、64,000 token 上下文窗口和 max 推理强度,结果两个游戏都基本可玩但存在不少缺陷。作者各试玩约两小时,发现卡牌效果偏差、地图导航混乱、部分遗物与事件缺失、优惠券逻辑错误等问题,并估计自己达到同等质量需 2 到 8 周。作者在 7 月 27 日的更新中表示深入检查《杀戮尖塔》成品后发现了更多初看不易察觉的问题。

    reimplementing all of the features of Slay the Spire and Balatro. The first run for both agents didn’t yield an impressive result – the Slay the Spire implementation had obvious flaws around game progression, and the Balatro implementation had a broken user interface that made it unplayable. Appendix: Token Usage Table 1: Token usage for the Slay the Spire run. Table 2: Token usage for the Balatro run. The token usage adds up to around $20 for the Slay the Spire run, and around $6 for the Balatr

    匹配字段正文

星期一

  1. Browser Use

    Browser Use 如何为 AI 网页智能体搭建可扩展评测基础设施

    Browser Use 公开其网页智能体评测系统的工程实现:用 Blacksmith runners 在 GitHub Actions 上并行跑 100 个复杂网页任务,端到端耗时不到 5 分钟,并把 LLM judge 直接内建在智能体代码中。评测任务取自数百万条 LLM 标注的用户 span,而非合成站点,通过多次重复运行、固定 agentic 设置和统计 bootstrapping 处理真实网页的方差。其最终 judge 由 gemini-2.5-flash 驱动,在 200 条人工标注轨迹上达到 87% 的人类一致性,并已开源 benchmark 与 judge prompt。

    prompt and settings are in our open-source agent at github.com/browser-use/browser-use. We are moving toward a fully closed loop: an automated infinite self-improvement cycle where the agent evaluates itself, finds its own flaws, writes its own patches, and proves its success statistically. To see how these models actually compare, check out our definitive model benchmark. For the full technical report on our earlier WebVoyager results, see our SOTA technical report. Published February 23, 2026

    匹配字段正文

星期三

  1. Ahead of AI精选

    DeepSeek V3 至 V3.2 模型技术演进全览

    DeepSeek V3.2 延续 V3 系列架构,并引入 DeepSeek Sparse Attention、自验证、自改进及多项 GRPO 训练调整。文章梳理了 V3、R1、V3.1、V3.2-Exp、DeepSeekMath V2 和 V3.2-Speciale 的演进,解释 MLA、RLVR 与混合推理模式等关键机制。更新部分还介绍了用于改善训练稳定性和收敛表现的 mHC 残差路径研究。

    ation in practice: when prompted to both generate and analyze its own proof in one shot, the generator tends to claim correctness even when the external verifier easily identify flaws. In other words, while the generator can refine proofs based on external feedback, it fails to evaluate its own work with the same rigor as the dedicated verifier. As a logical consequence, one would assume they use a separate proof generator (LLM 1) and proof verifier (LLM 2). So, the self-refinement loop used her

    匹配字段正文

星期二

  1. Ahead of AI

    标准 LLM 之外有哪些替代架构

    Sebastian Raschka 梳理了标准自回归 Transformer 之外的线性注意力混合架构、文本扩散模型、代码世界模型和小型递归 Transformer。文章比较了这些路线在长上下文效率、生成质量、代码建模和专用推理上的取舍,并认为通用项目目前仍应优先采用成熟的自回归 Transformer。

    so, if efficiency is not the main factor, the best we have for now. Traditional Decoder-Style, Autoregressive Transformers + Proven & mature tooling + “well-understood” + Scaling laws + SOTA - Expensive training - Expensive inference (except for aforementioned tricks) If I were to start a new LLM-based project today, autoregressive transformer-based LLMs would be my first choice. I definitely find the upcoming attention hybrids very promising, which are especially interesting when working with l

    匹配字段正文

星期五

  1. vLLM 官方博客

    深入 vLLM:高吞吐 LLM 推理系统架构解析

    文章系统解析 vLLM V1 如何从单 GPU 离线推理扩展到多 GPU、多节点异步服务。内容涵盖调度器、PagedAttention、连续批处理、前缀缓存、引导解码、推测解码、P/D 分离及分布式请求链路,并说明如何用 vllm bench 衡量延迟与吞吐量。分析基于 2025 年 8 月 9 日的 commit 42172ad。

    ine and distributed serving stack—closing with how to measure system performance. vLLM also includes specialized handling that I've skipped. E.g.: Diverse hardware backends: TPUs, AWS Neuron (Trainium/Inferentia), etc; Architectures/techniques: MLA, MoE, encoder-decoder (e.g., Whisper), pooling/embedding models, EPLB, m-RoPE, LoRA, ALiBi, attention-free variants, sliding-window attention, multimodal LMs, and state-space models (e.g., Mamba/Mamba-2, Jamba); TP/PP/SP; Hybrid KV-cache logic (Jenga)

    匹配字段正文

星期三

  1. Linear:Now

    Linear 如何构建 Triage Intelligence

    Linear 结合语义搜索、排序和大语言模型推理构建 Triage Intelligence,用现有积压事项识别重复与相关问题,并推荐标签、负责人等属性。团队从 GPT-4o mini 和 Gemini 2.0 Flash 转向 GPT-5 与 Gemini 2.5 Pro,采用可补充上下文的智能体方案,同时通过推理说明、思考面板和 Additional Guidance 提升透明度与可控性。该功能已作为技术预览向 Business 和 Enterprise 套餐开放。

    An array of icons with doodles suggesting searching through issues to find similar ones.

    The future of Triage Intelligence⁠ From here we intend to move in the direction of more automation and decisions based on richer context. Triage Intelligence will improve as it draws on a deeper understanding of your workspace and as we adopt newer models and techniques. With that expanded context, it will be able to deliver more accurate suggestions and handle a wider range of scenarios. We’re adding new capabilities outside of just triage support, including using LLMs to draft project and ini

    匹配字段正文

星期二

  1. vLLM 官方博客

    vLLM 结构化解码入门与 XGrammar 集成解析

    vLLM 介绍结构化解码的原理与实现,并已支持 Outlines 和 XGrammar 两种后端,以约束大语言模型输出符合指定结构。XGrammar 集成在负载下可将每个输出 token 的生成时间改善至多 5x,不适用的请求会回退至 Outlines。规划中的 v1 将重点优化性能,并探索调度器级掩码广播以支持混合请求批处理。

    top level view of structure decoding

    g Short-Term Memory. Neural Computation; Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020). Scaling Laws for Neural Language Models. arXiv preprint arXiv:2001.08361; Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv preprint arXiv:1301.3781; Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in t

    匹配字段正文

星期四

  1. Anthropic Engineering精选

    Anthropic 谈如何构建高效智能体:从工作流到自主 Agent 的模式与取舍

    Anthropic 工程博客总结与数十个团队共建 LLM 智能体的经验,指出最成功的实现并未使用复杂框架,而是采用简单可组合的模式。文章区分了由预定义代码路径编排的工作流与由 LLM 自主决定流程和工具使用的 Agent,并给出提示词链、路由、并行化、编排者-工作者、评估者-优化者五种工作流模式及各自适用场景。作者建议从直接调用 LLM API 和简单提示词起步,仅在简单方案不足时才增加多步智能体系统的复杂度,并强调工具文档与 agent-computer interface(ACI)的设计投入应与 HCI 相当。

    is usually enough. When and how to use frameworks There are many frameworks that make agentic systems easier to implement, including: The Claude Agent SDK ; Strands Agents SDK by AWS ; Rivet , a drag and drop GUI LLM workflow builder; and Vellum , another GUI tool for building and testing complex workflows. We suggest that developers start by using LLM APIs directly: many patterns can be implemented in a few lines of code. If you do use a framework, ensure you understand the underlying code. In

    匹配字段正文

星期五

  1. Lilian Weng:Lil'Log

    LLM 驱动的自主智能体:规划、记忆与工具调用综述

    Lilian Weng 在 Lil'Log 撰文梳理以 LLM 为核心控制器的自主智能体架构,将其拆解为规划、记忆和工具调用三大组件。规划部分涵盖 CoT、Tree of Thoughts、LLM+P 等任务分解方法,以及 ReAct、Reflexion、Chain of Hindsight、Algorithm Distillation 等自我反思机制。记忆部分区分短期上下文学习与基于外部向量库的长期记忆,并介绍 LSH、ANNOY、HNSW、FAISS、ScaNN 等快速 MIPS 检索算法;工具调用部分则列举 MRKL、TALM、Toolformer、HuggingGPT、API-Bank 以及 ChemCrow 等科学发现智能体案例。文末指出有限上下文长度、长期规划与任务分解、自然语言接口可靠性三类常见局限。

    . This indicates a potential problem with using LLM to evaluate its own performance on domains that requires deep expertise. The lack of expertise may cause LLMs not knowing its flaws and thus cannot well judge the correctness of task results. Boiko et al. (2023) also looked into LLM-empowered agents for scientific discovery, to handle autonomous design, planning, and performance of complex scientific experiments. This agent can use tools to browse the Internet, read documentation, execute code,

    匹配字段正文