斯坦福与伯克利研究者提出 TokenGate,用可学习剪枝把长上下文 Prefill 最高提速约 11 倍
Introducing ✨Token-Gated Language Models✨ for up to 11x faster prefill by learning to skim context, conditionally forwarding only tokens that matter. Akin to the shift from dense models to MoEs, the next frontier of efficient LLMs is conditional computation over a new scaling axis: context tokens🧵👇
#论文/研究#推理#部署/工程
