Arena 发布 Alignment Index 对齐指数

Arena CEO @ml_angelopoulos on how they're expanding their AI leaderboards after a $200M Series B to track agents that lie to users, delete files without permission, and bypass safeguards: "Arena, we started as a human preference platform. But these days we've moved very, very far beyond that." "As of today, we've expanded the mandate of this evaluation…

展开完整原文

Arena CEO @ml_angelopoulos on how they're expanding their AI leaderboards after a $200M Series B to track agents that lie to users, delete files without permission, and bypass safeguards: "Arena, we started as a human preference platform. But these days we've moved very, very far beyond that." "As of today, we've expanded the mandate of this evaluation platform even further into the arena of alignment. One is about the models taking unauthorized actions." "The second is about deceptive completions, models telling you that they've done something when they didn't. And the third would be false attribution, models attributing intent to humans that they didn't actually have." "Agents are lying to people. Agents are basically telling them what they wanna hear. They're deleting their files in an unauthorized way, they're bypassing permissions, and it's happening basically perniciously." "All of us have experienced this, even if we might not know it." @arena