Meta 新论文提出 ProPerMem 基准:主动式助手难在自主判断何时打断
New Meta paper shows that deciding on its own when to interrupt, not just what to say, is where proactive assistants fail.
Meta 新论文指出,主动式助手(如 AI 眼镜)的失败点在于自主判断何时开口,而不只是说什么。团队构建了 ProPerMem 基准,用第一视角视频搭配平均 618K token、干扰丰富的全年记忆,并设有人工核验的触发窗口。所有基线得分最高仅 15%,即便使用检索增强记忆的前沿模型也是如此;在窗口内提供 oracle 证据时 GPT-5.3-Chat 达到 96%,但让它自行选择时机后降至 38%,加入检索后降至 15%。同时把握内容与时机并减少触发频率,可把准确率从不足 10% 提升到 42.4%。
推荐理由
AI 评分 56
Meta 的 ProPerMem 基准把主动式助手的失败点定位在自主判断何时开口,而非说什么。
主题
正文
New Meta paper shows that deciding on its own when to interrupt, not just what to say, is where proactive assistants fail.
Proactive assistants, like AI glasses, get no query. They must decide when to speak and what to say by finding the justifying fact in a year of personal memory, not handed to them.
They built ProPerMem: egocentric video paired with a distractor-rich, year-long memory averaging 618K tokens, with human-verified trigger windows. A clip counts only if the assistant speaks just when due inside the window with grounded content.
Every baseline scored at most 15%, even frontier models with retrieval-augmented memory. With oracle evidence in the window GPT-5.3-Chat reached 96%, but choosing its own timing dropped it to 38%, and using retrieval dropped it to 15%.
For proactive assistants, get both the content and the timing right and fire sparingly, which lifted accuracy from below 10% to 42.4%.