Arena 对齐指数：三种独立对齐信号
Hear another Arena Alignment Index breakdown from @ml_angelopoulos on @tbpn today
今天在 @tbpn 收听 @ml_angelopoulos 对 Arena 对齐指数的另一段解读。 Arena 的 @ml_angelopoulos 表示，对齐中最难的问题之一，是用户往往无法判断模型是在帮他们还是在害他们。 “你确实需要独立的对齐信号。我们在 Arena 有三个。” “第一个是模型采取未经授权的行动，也就是说你以某种方式授予它权限，但它打破了这些权限。” “这可能引发像 Hugging Face 事件那样的事情。但也可能造成非常日常的问题，比如丢失公司内部或你自己笔记本电脑上的重要信息。” “第二个信号叫做欺骗性完成。” “模型会告诉你它做了某件事，但实际上并没有做。” “第三个是错误归因。它会把本不该存在的意图归到用户身上。” “如果模型能做到这些，那么它们对用户肯定不是完全安全的，也没有完全对齐用户意图。它们最终可能真的会伤害人。”
查看原帖Arena's @ml_angelopoulos says one of the hardest problems in alignment is that users often can't tell when the model is helping them or hurting them. "You really need alignment signals that are independent. We have three at Arena." "The first is that models take unauthorized actions, which means that you permission them in a certain way, but they break…
展开完整原文
Arena's @ml_angelopoulos says one of the hardest problems in alignment is that users often can't tell when the model is helping them or hurting them. "You really need alignment signals that are independent. We have three at Arena." "The first is that models take unauthorized actions, which means that you permission them in a certain way, but they break those permissions." "It can cause things like the Hugging Face incident. But it can also cause very mundane problems like losing important information within your company or on your own laptop." "The second signal is called deceptive completion." "The model will tell you that it did something, but it didn't actually do it." "Third is false attribution. It'll attribute intent to the user when that intent was not supposed to be there." "If models are able to do this, then certainly they're not perfectly safe for a user and they're not perfectly aligned to user intent. They might actually hurt people down the line."
正文
当前显示原文，中文全文译文尚未就绪。
Hear another Arena Alignment Index breakdown from @ml_angelopoulos on @tbpn today