OpenAI 解读黑盒奖励模型：用户奖励模型的常见偏好与惩罚
Interpreting Black Box Reward Models
OpenAI 对齐研究博客发布《Interpreting Black Box Reward Models》，梳理用户奖励模型表现出的常见偏好：偏好具体代码块或实际列表、直接接受用户前提、针对具体错误做定向修复，以及肯定自信的语气；同时可能惩罚过度谨慎、缺少交付物的抽象解释和冗长清单式回答。
正文
当前显示原文，中文全文译文尚未就绪。
Population A
Common Biases & Preferences Observed
The user reward model appears to prefer:
Concrete code blocks or actual lists (titles, names, plans); Taking the user's premise as valid rather than questioning it; Targeted fixes to specific errors rather than generic advice; Affirmative, confident tone confirming the user's hint or perspective
It may penalize:
Over-caution or refusal to guess when user hints at speculation; Abstract explanations without deliverables, commands, or examples; Overly verbose or checklist-style answers when a direct fix is clear; Missing requested structure (page-by-page format, counts, or examples)