Chroma 研究:输入 token 增加如何影响 LLM 性能
Chroma Research 发布 Context Rot 技术报告,在保持任务复杂度不变、只改变输入长度的受控实验中测试 18 个模型,发现性能随输入变长而持续下降,且下降方式并不均匀。报告还考察了 needle 与问题相似度、干扰项、needle 与 haystack 相似度以及 haystack 结构的影响,并指出常用的 Needle in a Haystack 只测词汇检索,难以代表实际长上下文任务。研究同时给出可复现结果的完整代码库。

ts to investigate the influence of these factors: Needle-Question Similarity We compute the cosine similarity between needle-question pairs using embeddings. For robustness, we average across five embedding models: text-embedding-3-small, text-embedding-3-large, jina-embeddings-v3, voyage-3-large, and all-MiniLM-L6-v2. We measure how model performance is impacted by needle-question similarity as input length increases. Impact of Distractors Taking a high-similarity needle-question pair, we write
匹配字段正文

