基准测试的偏差：只能覆盖通用智能的一小部分
not exactly - you can benchmark certain aspects of reliability (e.g. consistency through pass^k)
AI 导读
不完全是——你可以对可靠性的某些方面做基准测试（例如通过 pass^k 衡量一致性） 我对基准测试的疑虑在于偏差。按定义，它们只能捕捉通用智能地图的一小部分，而且很容易为了已知的那部分地图过度优化，牺牲掉其余部分
正文
当前显示原文，中文全文译文尚未就绪。
not exactly - you can benchmark certain aspects of reliability (e.g. consistency through pass^k)
my issue with benchmarks is bias. by definition, they only capture a small part of the map of general intelligence, and it's very easy to overoptimize for the known map at the expense of the rest