Nathan Lambert 评基准测试：开源模型未衰，AA 指数曾高估
Open models' demise has been greatly exaggerated, but the previous AA index did flatter them. As an industry we need more specific intelligence indices, as LLMs
AI 导读
Nathan Lambert 认为开源模型的衰落被夸大，但此前的 AA 指数确实对它们有所高估。他主张业界需要更细分的智能指数，因为 LLM 被用于差异极大的任务；这是他针对基准测试行业的元评测。
正文
Open models' demise has been greatly exaggerated, but the previous AA index did flatter them. As an industry we need more specific intelligence indices, as LLMs are used in vastly different tasks.
My meta-review on the benchmarking industry. https://t.co/7lT09ijAqj