AWS 与 HSBC 论文提出 Double Ratchet:评分器随智能体同步演化
Self-improving agents have always needed a human to write the grader.
AWS 与 HSBC 的一篇新论文提出 Double Ratchet,让评分器随智能体技能同步演化,不再需要人工编写评分标准。该方法用 10 个标注样本筛选评分器版本,且只以与标注样本一致为准,不以抬高智能体得分为目标。在代码、SQL 和报告任务上,它保留了真实标注或评分细则所带来增益的 88% 到 110%;移除防护机制后,评分器放行了 97% 到 100% 的输出,但训练技能的效果相当。
正文
当前显示原文,中文全文译文尚未就绪。
Self-improving agents have always needed a human to write the grader.
A new AWS + HSBC paper removes that step: the grader evolves alongside the agent from 10 labeled examples and trains it about as well as ground truth.
Double Ratchet evolves the grader alongside the agent's skills, building it from small, mostly rule-based checks for 1 failure type each. A grader version is chosen for agreeing with 10 labeled examples, never for raising the agent's score.
It kept 88% to 110% of the gain that ground truth or a rubric gave the same skill loop, across code, SQL, and reports. With the safeguards removed, the grader passed 97% to 100% of outputs yet trained skills as well.