←新闻详情

Rohan Paul更新于 10月21日 22:46

DigitalOcean 推出 Signals 与 Simulations,捕捉智能体静默失败

Agent failures often pass every health check. A tool call hits the wrong database region, returns 200, and the user gets a slower, worse answer. Nothing alerts.

AI 导读

DigitalOcean 发布 Agent Learning Services 与 Agent Data Services,其中 Signals 用低成本规则而非 AI 模型监控实时会话,识别用户重复提问、要求转人工或工具循环重试等异常信号,再交给 Simulations 用模拟用户回放这些会话,在修复上线前验证整段对话是否正常。此前团队需要付费让 LLM 给每段对话打分、人工阅读记录或凭空构造测试用例,现在生产环境中的真实失败会自动成为测试集。

推荐理由

AI 评分 58

原文给出 DigitalOcean 用规则信号加回放模拟定位智能体静默失败的做法,可对照现有评测流程理解其差异。

主题

标签

引用

DigitalOcean

@digitalocean

Catch it at the first user, not the four-hundredth. A new user’s bank link fails, the agent says “try again later,” and nothing shows up in the logs. Watch a team find the bug and fix it before anyone else hits it, with Agent Learning Services and Agent Data Services, launching today.

查看原帖

正文

当前显示原文,中文全文译文尚未就绪。

Agent failures often pass every health check. A tool call hits the wrong database region, returns 200, and the user gets a slower, worse answer. Nothing alerts.

DigitalOcean’s new tools flag agent sessions that fail silently, then replay them against a fix before release.

Because, the clue behind an agent bug is often sitting in something nobody can search.

Signals watches every live session with cheap rules, not an AI model, that look for human trouble signs such as a user repeating themselves, asking for a human, or a tool being retried in a loop, then passes those sessions to Simulations, which replays them with a fake user against your fixed agent to check the whole conversation works before the fix ships.

Until now teams had to pay an LLM to grade every conversation, read transcripts by hand, or invent test cases from imagination, while here the real failures from production automatically become the test set.

来源

Rohan Paul