←搜索结果

搜索“OpenAI”

按匹配度 · 共 148 条结果 清除搜索

星期二

  1. vLLM 官方博客精选

    vLLM 上 Kimi K2 工具调用准确率的深度调试

    vLLM 与 Kimi 团队修复了 Kimi K2 的关键工具调用兼容问题，使成功解析的调用数从 218 个增至 971 个，提升 4.4x。问题涉及缺失 add_generation_prompt、空 content 的结构转换，以及历史工具调用 ID 不符合 functions.func_name:idx 格式。修复后仍出现 316 次 schema_validation_error_count，团队正协作将用于约束工具选择的 Enforcer 组件集成到 vLLM。

    ken IDs sent to the model is the only way to be certain. While I didn't need to resort to this for the issues above, it's a critical tool in the toolbox. Techniques like using the OpenAI-compatible API to return token IDs can be a lifesaver. For those interested, we also highlighted this in our Agent Lightning post; Understand Framework Design Philosophy: vLLM's strict handling of kwargs is not a bug, but a deliberate security choice. Understanding these design decisions helps in quickly identif

    匹配字段正文

    #engineering#教程/实践#MCP/工具调用#部署/工程

星期日

  1. Ahead of AI

    从零理解 LLM 评估的四种主要方法

    文章讲解评估 LLM 的四种主要方法，包括多项选择、验证器、排行榜和 LLM 评审，并配有从零代码实现。不同方法分别侧重知识准确率、可验证任务、用户偏好和整体回答质量，各有成本、偏差与适用范围，因此模型评估应组合多种方法，并使用贴合目标领域的数据。

    on, we will implement the judge-based evaluation shown in Figure 12 using the Ollama API in Python. Specifically, we will use the 20-billion parameter gpt-oss open-weight model by OpenAI as it offers a good balance between capabilities and efficiency. For more information about gpt-oss, please see my From GPT-2 to gpt-oss: Analyzing the Architectural Advances article: From GPT-2 to gpt-oss: Analyzing the Architectural Advances 4.1 Implementing a LLM-as-a-judge approach in Ollama Ollama is an eff

    匹配字段正文

    #newsletter#教程/实践#评测/基准#推理

星期五

  1. vLLM 官方博客

    深入 vLLM：高吞吐 LLM 推理系统架构解析

    文章系统解析 vLLM V1 如何从单 GPU 离线推理扩展到多 GPU、多节点异步服务。内容涵盖调度器、PagedAttention、连续批处理、前缀缓存、引导解码、推测解码、P/D 分离及分布式请求链路，并说明如何用 vllm bench 衡量延迟与吞吐量。分析基于 2025 年 8 月 9 日的 commit 42172ad。

    DP coordinator: sending wave triggers, polling LB state, and handling dynamic scaling requests Finally, the main server process creates a FastAPI app and mounts endpoints such as OpenAIServingCompletion and OpenAIServingChat, which expose /completion, /chat/completion, and others. The stack is then served via Uvicorn. So, putting it all together, here's the full request lifecycle! You send from your terminal: What happens next: The request hits OpenAIServingCompletion's create completion route

    匹配字段正文

    #engineering#教程/实践#部署/工程#开源生态

星期一

  1. Vapi Blog

    什么是 Gemma 3？Google 开源权重模型的能力与部署解析

    Vapi 博客介绍 Google 于 2025 年发布的开源权重模型 Gemma 3，它基于 Gemini 2.0 研究，支持文本、图像和视觉输入，可在单 GPU/TPU 上运行，支持 140+ 语言和最高 128k token 上下文，并提供 1B、4B、12B、27B 四种尺寸。文章称 Gemma 3 在人类偏好评测中优于 Llama3-405B 和 DeepSeek-V3，27B 版本在 LMArena Elo 榜得 1338 分，高于 DeepSeek-V3 的 1318 和 o3-mini 的 1304。经量化感知训练后，1B 模型显存占用最低 861 MB（INT4），27B 为 19.9 GB（INT4）；该模型已内置在 Vapi 控制台的 LLM 下拉菜单中。

    apabilities for complex tasks; 27B model: Enterprise-grade performance In early evaluations, Gemma 3 consistently outperforms larger competitors like DeepSeek-V3, Llama3-405B, and OpenAI's o3 mini while requiring significantly fewer computational resources. » Speak to a Gemma 3-powered digital voice assistant. To fully understand what Gemma 3 is, it helps to see how it evolved. Gemma 2 was primarily text-focused, handling context windows of 8k to 32k tokens with support for around 20 languages,

    匹配字段正文

    #company#产品更新#多模态#端侧

星期一

  1. Johann Rehberger / Embrace The Red

    ChatGPT 记忆与聊天历史功能如何运作：一次系统提示词实测

    Johann Rehberger 用 ChatGPT o3 和两个独立账号实测了 ChatGPT 的记忆与聊天历史功能，发现系统提示词中新增了 Model Set Context、Assistant Response Preferences、Notable Past Conversation Topics Highlights、Helpful User Insights、Recent Conversation Content 和 User Interaction Metadata 等区块。其中 Recent Conversation Content 保存约 40 条最近对话，包含时间戳、对话摘要和用户输入的全部消息，但不含 ChatGPT 的回复；作者尝试用一年前的特定话题提问，ChatGPT 无法检索到这些历史对话，说明该功能并非全文搜索或 RAG 检索。作者还指出用户无法查看 ChatGPT 具体存储了哪些信息，认为应提高透明度并允许删除条目。

    Recently OpenAI added an additional memory feature called “chat history”, which allows ChatGPT to reference past conversations. The details of the implementation are not known. The documentation highlights that: “It uses this to learn about your interests and preferences, helping make future chats more personalized and relevant.” I decided to spend some time to figure out how it works. Update: Video Tutorial Added Based on the interest in this post, I’ve also created a video tutorial. Enjoy and

    匹配字段正文

    #analysis#安全/对齐#OpenAI#Agent

星期二

  1. vLLM 官方博客

    vLLM 结构化解码入门与 XGrammar 集成解析

    vLLM 介绍结构化解码的原理与实现，并已支持 Outlines 和 XGrammar 两种后端，以约束大语言模型输出符合指定结构。XGrammar 集成在负载下可将每个输出 token 的生成时间改善至多 5x，不适用的请求会回退至 Outlines。规划中的 v1 将重点优化性能，并探索调度器级掩码广播以支持混合请求批处理。

    top level view of structure decoding

    where structured decoding comes in. It enables LLMs to generate outputs that follow a desired structure while preserving the non-deterministic nature of the system. Companies like OpenAI have recognized this need, implementing features like JSON mode to constrain6 the output format. If you have built with these functionalities before (such as agentic workflows, function calling, coding assistant), chances are you are using structured decoding under the hood. Guided decoding is to LLMs what valid

    匹配字段正文

    #engineering#教程/实践#部署/工程#Agent

星期五

  1. Lilian Weng：Lil'Log

    LLM 驱动的自主智能体：规划、记忆与工具调用综述

    Lilian Weng 在 Lil'Log 撰文梳理以 LLM 为核心控制器的自主智能体架构，将其拆解为规划、记忆和工具调用三大组件。规划部分涵盖 CoT、Tree of Thoughts、LLM+P 等任务分解方法，以及 ReAct、Reflexion、Chain of Hindsight、Algorithm Distillation 等自我反思机制。记忆部分区分短期上下文学习与基于外部向量库的长期记忆，并介绍 LSH、ANNOY、HNSW、FAISS、ScaNN 等快速 MIPS 检索算法；工具调用部分则列举 MRKL、TALM、Toolformer、HuggingGPT、API-Bank 以及 ChemCrow 等科学发现智能体案例。文末指出有限上下文长度、长期规划与任务分解、自然语言接口可靠性三类常见局限。

    d on whether a newly added API call annotation can improve the quality of model outputs. See more details in the “External APIs” section of Prompt Engineering. ChatGPT Plugins and OpenAI API function calling are good examples of LLMs augmented with tool use capability working in practice. The collection of tool APIs can be provided by other developers (as in Plugins) or self-defined (as in function calls). HuggingGPT (Shen et al. 2023) is a framework to use ChatGPT as the task planner to select

    匹配字段正文

    #analysis#教程/实践#Agent#推理

星期三

  1. Lilian Weng：Lil'Log

    Lilian Weng 长文梳理提示词工程方法：从零样本到 CoT 与工具调用

    Lilian Weng 发布提示词工程综述，系统梳理零样本、少样本、指令提示、CoT、自洽采样、检索增强与外部工具调用等方法，并指出提示词效果因模型而异，需要大量实验和启发式调整。文章还汇总了示例选择与排序、自动提示词生成、Tree of Thoughts、Toolformer 等方向的研究，并附 26 篇论文引用。

    s an input for another tool) or in an interactive way (i.e. adopt API response after human selection). Both are interesting future directions to expand the model for. Cited as: Or OpenAI Cookbook has many in-depth examples for how to utilize LLM efficiently; LangChain, a library for combining language models with other components to build applications; Prompt Engineering Guide repo contains a pretty comprehensive collection of education materials on prompt engineering; learnprompting.org; Prompt

    匹配字段正文

    #analysis#教程/实践#推理#数据/训练