SanSi 循环复用层让 Ouro-1.4B 决策准确率升至 72%
This paper finds that small decision models get far more accurate by rerunning the same layers several times before giving an answer.
论文发现，让小型决策模型在给出答案前多次重跑同一批层可显著提升准确率。SanSi 循环运行预训练即复用层的 Ouro-1.4B，在 8 次循环中每次读取选项概率，在 10,027 条测试决策上得分 72.0%，同结构无循环模型为 58.4%，但计算量达 7.7 倍。在长于训练分布的 liar chains 上，SanSi 达 72.6%，而 4.2B 单次前向模型仅 50.0%（随机水平）。
正文
当前显示原文，中文全文译文尚未就绪。
This paper finds that small decision models get far more accurate by rerunning the same layers several times before giving an answer.
Many LLM calls only need a choice, like spam or not spam. Typed decision models answer these in a single pass, which fails when a decision needs several dependent steps.
SanSi loops Ouro-1.4B, a model pretrained to reuse its layers, and reads option probabilities after each of 8 loops. Every loop is trained to answer, so the same model can stop anywhere from 1 to 8 loops.
On 10,027 test decisions, SanSi scored 72.0% versus 58.4% for a same-shape model without loops, at 7.7x the compute. On liar chains longer than any seen in training, it reached 72.6% while a 4.2B single-pass model scored 50.0%, the chance level.
If memory limits your decision-style calls, try a looped small model before moving to a model 3x larger.