规划、拆分、攻克：弱模型如何擅长长上下文任务
Plan, divide, and conquer: How weak models excel at long context tasks
Together AI 的研究发现，采用精心设计的“分而治之”框架后，Llama-3-70B、Qwen-72B 等较弱模型在长上下文检索、问答和摘要任务中可媲美或超过单次处理全文的 GPT-4o。该方法通过改进聚合提示来减少噪声，但只适合跨文本块依赖适中的任务；需要关联远距离线索时，拆分会破坏必要上下文。
正文
当前显示原文，中文全文译文尚未就绪。
TL;DR
The Intuition: Don't ask one genius to read a library in an hour. Ask ten interns to read one book each.
Our research, "When Does Divide and Conquer Work for Long Context LLM?" (ICLR 2026), introduces a framework to study this. We found that smaller models using a strategic "Divide & Conquer" design can match or beat GPT-4o single-shot on long context tasks.
[ paper, ICLR 2026 ] [ code ]
Modern LLMs increasingly support massive context windows like 128K, 200K, even 1M+ tokens. This theoretically unlocks powerful use cases like analyzing entire codebases or summarizing full books in a single prompt.
However, the promise of "just throw everything into one prompt" frequently fails in practice. As context length grows, performance degrades in unexpected ways. “Divide and Conquer” turns out to be an attractive solution to long context tasks, as shown in the figure below.
In our paper, we found that weaker models using a carefully designed "Divide & Conquer" framework can match or beat GPT-4o single-shot on long context tasks.
How can we fully leverage the power of such a “Divide and Conquer” approach for long context tasks? We break the challenge down into three distinct sources of noise:
Naive "MapReduce" approaches often fail because of Aggregator Noise i.e. the final answer is messy or inconsistent because the manager lacks context. Our framework predicts that if you reduce this noise through clearer instructions, you can unlock the power of weaker models.
The "2nd Smallest Number" Example: Consider a task where workers must help find the 2nd smallest number in a massive list. If you naively ask workers to "Find the 2nd smallest number" in their specific chunk, the final manager will fail because the global 2nd smallest number might be the smallest number in a specific chunk.
By simply adjusting the prompt to account for this aggregation noise, we ensure the Manager has the right data to calculate the global answer.
We tested this framework implementation across diverse tasks (retrieval, QA, summarization, etc.). As shown below, a weaker model using this framework (e.g., Llama-3-70B or Qwen-72B) outperforms GPT-4o reading the whole text in one shot. While the single-shot model (red dashed line) degrades as context length increases, the D&C models maintain high performance.
From the engineering perspective, this framework offers three massive practical benefits:
This method is not a silver bullet. It works best for tasks like QA, Retrieval, and Summarization where cross-chunk dependency is moderate. The benefits end when Task Noise dominates . If your task requires tracking a subtle clue from Page 1 that connects to Page 100 (like the "Dialogue Character Inference" task in our paper), the "Divide" step breaks the necessary context. In these high-synergy cases, the 'genius' model reading the whole library is still the only way to go. Please find more details in our paper.