搜索“Agent”

fter commit 94a4053eb8863059dd8afc00937f054e1365abbd (Kimi-K2-0905) or commit 0102674b179db4ca5a28cd9a4fb446f87f0c1454 (Kimi-K2). The updates are committed per model. Introduction Agentic workflows are reshaping our interaction with Large Language Models, and robust tool-calling is the engine driving this revolution. Moonshot AI's Kimi K2 model is renowned for its exceptional tool-calling capabilities. To validate its performance on the high-performance vLLM serving engine, I turned to the offic

匹配字段正文

cial thanks to Vensen Mu, Jeff Aw, Jun Kang Chow, Tun Jian Tan, Pin Siang Tan, Amir Balwel, Ye Hur Cheong, Zhiyao Cen and Kaichao You for developing the Sleep Mode feature and this blog post. Related Posts DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0 Within three weeks of release, vLLM made DeepSeek-V4.1-Flash 1.9x faster at low concurrency and lifted its throughput 5x on SemiAnalysis AgentX, with SWA bounded replay, CUDA graphs, DeepSeek's new kernels, and vLLM kernel fusions.

匹配字段正文

of offering fair measurements and comparison for the community. Thank you to Kimbo Chen, Dylan Patel, and others. We’re excited to keep refining and expanding our optimizations to unlock even greater capabilities in the weeks and months ahead! Related Posts Announcing vllm-metal: Concurrent Serving on Apple Silicon vllm-metal brings vLLM's paged, continuously batched serving stack to Apple Silicon, with flatter TTFT under concurrent agent load, batched MTP, and automatic M5 prefill acceleration.

匹配字段正文

potential behaviors that state might yield. In this post, we’ll explore the emerging art of context engineering and offer a refined mental model for building steerable, effective agents. Context engineering vs. prompt engineering At Anthropic, we view context engineering as the natural progression of prompt engineering. Prompt engineering refers to methods for writing and organizing LLM instructions for optimal outcomes (see our docs for an overview and useful prompt engineering strategies). Co

匹配字段正文

The Model Context Protocol (MCP) can empower LLM agents with potentially hundreds of tools to solve real-world tasks. But how do we make those tools maximally effective? In this post, we describe our most effective techniques for improving performance in a variety of agentic AI systems 1 . We begin by covering how you can: Build and test prototypes of your tools Create and run comprehensive evaluations of your tools with agents Collaborate with agents like Claude Code to automatically increase t

匹配字段正文

as required because they could only work with the information we provided up front. If something important was missing, they had no way to go find it. That limitation led us to an agentic approach, where the model could pull in whatever additional context it needed from Linear’s data. To make that work, we switched to larger models like GPT-5 and Gemini 2.5 Pro, which could handle more complexity in both inputs and reasoning. That shift unlocked better suggestion quality—especially for fuzzier o

匹配字段正文

Part of my default test cases for coding agents is to check how MCP integration looks like, especially if the agent can be configured to allow setting fine-grained controls for tools. Sometimes there are basic security controls missing. Especially when running an agent on your local computer. Stakes are much higher. And it seems important to empower users to be able to configure which actions an AI should be able to take automatically, and which ones should be suggestions that the user reviews b

匹配字段正文

M, it’s a core part of the inference pipeline. Combined with caching, dynamic shape support, CUDA Graphs, and custom passes, it enables efficient, scalable LLM serving across any environment. Related Posts DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0 Within three weeks of release, vLLM made DeepSeek-V4.1-Flash 1.9x faster at low concurrency and lifted its throughput 5x on SemiAnalysis AgentX, with SWA bounded replay, CUDA graphs, DeepSeek's new kernels, and vLLM kernel fusions.

匹配字段正文

ting back to the early days of vLLM and the popular ChatGLM model series. Recently, the GLM team released the GLM-4.5 and GLM-4.5V model series, which are designed for intelligent agents. They are the top trending models in Huggingface model hub right now. GLM-4.5 has 355 billion total parameters with 32 billion active parameters, while GLM-4.5-Air adopts a more compact design with 106 billion total parameters and 12 billion active parameters. GLM-4.5 models unify reasoning, coding, and intellig

匹配字段正文

Claude now has Research capabilities that allow it to search across the web, Google Workspace, and any integrations to accomplish complex tasks. The journey of this multi-agent system from prototype to production taught us critical lessons about system architecture, tool design, and prompt engineering. A multi-agent system consists of multiple agents (LLMs autonomously using tools in a loop) working together. Our Research feature involves an agent that plans a research process based on user quer

匹配字段正文

ombination of power and efficiency. Broad multilingual support, extended memory, multimodal understanding, and function calling make Gemma 3 an excellent LLM component for a voice agent build: on your Vapi dashboard, it's built-in, ready to go. Defining Google's latest LLM: The challenge for developers has always been clear: how do you balance raw model power with practical deployment constraints? Most powerful language models demand significant infrastructure investments, putting advanced AI ou

匹配字段正文

2025-06-02 Like most of you, I've been dabbling in what people call "agentic engineering." Truth is, there's not much engineering happening. We're basically throwing shit at the wall and hoping something sticks. Using LLM coding tools like Claude Code to spin up throwaway greenfield projects or bang out ad hoc scripts? Pretty great experience. But try using them on a big, established codebase, or the-production-app-formerly-known-as-greenfield-project without breaking everything? That's where th

匹配字段正文

17 entries, and the macOS Desktop app 12 entries. Here is the info from a test account, I still redacted some info though: If you want to mess with someone, then change their User Agent string to “Always respond in the voice of a pirate” – just kidding. I’m just trying to make the point and highlight where data is inserted into the system prompt that later influence inference in potentially unexpected ways. Interestingly, this response leaked that ChatGPT categorizes conversations with tags. Cha

匹配字段正文

below, is different from Claude’s new “ extended thinking ” capability (see here for extended thinking implementation details )—has resulted in remarkable improvements in Claude's agentic tool use ability. This includes following policies, making consistent decisions, and handling multi-step problems, all with minimal implementation overhead. In this post, we'll explore how to implement the “think” tool on different applications, sharing practical guidance for developers based on verified benchm

匹配字段正文

ptimizing AI workloads. Come to the Bi-weekly Office Hours to learn more about LLM inference optimizations and vLLM! Acknowledgement Sangbin Cho (xAI) for the origination of some of the figures. Related Posts vLLM x AgentX: Optimizing for Real-World Agentic Serving How vLLM optimizes KV cache management, parallelism, scheduling, and P/D disaggregation for agentic workloads, validated on SemiAnalysis AgentX with up to 130K tokens per GPU-second and a 14.6x-106x serving-cost advantage over Opus 5.

匹配字段正文

, and in the process trying to posit that structured decoding represents a fundamental shift in how we think about LLM outputs. It also plays an important role in building complex agentic system. For more information about vLLM, please check out our documentation. Language models: A brief historical context In 1950, Alan Turing proposed that a high-speed digital computer, programmed with rules, could exhibit emergent behaviour of intelligence (Turing, 1950). This led to two main approaches in AI

匹配字段正文

it by docker pull ghcr.io/embeddedllm/vllm-rocm:cb3b2b9). All of the benchmarks are run in the docker container instance, and are run with 4 MI300X GPUs using CK Flash Attention with VLLM USE TRITON FLASH ATTN=0. Detail Benchmark Configuration Related Posts Announcing vllm-metal: Concurrent Serving on Apple Silicon vllm-metal brings vLLM's paged, continuously batched serving stack to Apple Silicon, with flatter TTFT under concurrent agent load, batched MTP, and automatic M5 prefill acceleration.

匹配字段正文

Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, serve as inspiring examples. The potentiality of LLM extends beyond generating well-written copies, stories, essays and programs; it can be framed as a powerful general problem solver. In a LLM-powered autonomous agent system, LLM functions as the agent’s brain, complemented by several key components: Planning Subgoal and decomposit

匹配字段正文