Anthropic 为 Claude 加入自评估安全机制
We’ve trained Claude to flag potential misuse in conversations, a technique we call self-assessment. Right after it responds, Claude gives a preliminary safety
Anthropic 训练 Claude 在回复后对对话给出初步安全评估，这一技术称为 self-assessment，用于标记潜在滥用。该机制自 6 月启用以来，已发现其他防护措施未能覆盖的滥用行为。Anthropic 表示，由于 Claude 已读过对话，自评估效率很高，且不同于针对固定危害类别的分类器，它依托模型自身判断，会随模型改进而更准确。
推荐理由
AI 评分 55
Anthropic 说明了自评估机制如何借助模型自身判断补充现有分类器，读者可了解其安全防护思路的差异。
主题
正文
We’ve trained Claude to flag potential misuse in conversations, a technique we call self-assessment. Right after it responds, Claude gives a preliminary safety assessment of the conversation.
Since we began using self-assessment in June, it has surfaced misuse that our other safeguards weren’t designed to catch.
Because Claude has already read the conversation, self-assessment is very efficient. And unlike classifiers, which target a discrete set of harms, it draws on Claude’s own judgment, so it gets more accurate as our models improve.
Read more on our new Safeguards blog: https://t.co/18j6zdHub4