Jev 验证器成本低 187 倍，误批率却高于 GPT-6 Astra
This paper finds that a cheap verifier can match a frontier LLM's accuracy at checking RAG claims, while letting more unsupported claims slip through.
论文在 495 条人工标注 claim 上对比 Jev 与 GPT-6 Astra：Jev 准确率仅差 0.6 个百分点，成本低 187 倍，但对无证据支持 claim 的放行率达 23.2%，Astra 为 14.1%。Jev 先答、其余转 Astra 的级联虽降低账单，但随机把同等比例 claim 交给 Astra 也能省下相近成本。作者建议 RAG 产品选 claim 检查器时重点测误批率，而非只看总体准确率。
正文
This paper finds that a cheap verifier can match a frontier LLM's accuracy at checking RAG claims, while letting more unsupported claims slip through.
RAG pipelines keep asking whether evidence supports a claim, a simple yes or no. Most teams get that answer from a text-generating LLM at generation prices.
They tested Jev, which returns only a label and a probability, against GPT-6 Astra on 495 human-labelled claims. They also built a cascade where Jev answers confident claims and passes the rest to Astra.
Jev landed within 0.6 points of Astra's accuracy at 187 times lower cost, but it approved 23.2% of unsupported claims against Astra's 14.1%. The cascade also cut Astra's bill, yet sending the same share of claims to Astra at random saved about as much.
Before picking a claim checker for a RAG product, measure how often it approves unsupported claims, since headline accuracy hides that.