让大模型学会自动判断何时停止读取上下文，节省算力还更准。
Knowing When to Stop: Efficient Context Processing via Latent Sufficiency Signals
- 通过分析注意力头，发现模型内部有可检测的“信息足够”信号。
- 在六大数据集上平均减少1.33倍输入token，准确率提升3.4%。
- 大模型能自判断，小模型需外加探测，适合高效推理场景。
大型语言模型（LLMs）通常无差别处理全部输入上下文，当问题所需信息集中在局部时效率低下。本文提出动态上下文截断方法，使模型在获取足够任务相关信息后自动终止处理。通过对模型内部结构的分析，我们发现特定注意力头天然编码了“充分性信号”，可通过轻量级分类器检测，从而预测关键信息是否已处理完毕。这揭示了一种新效率范式：模型内部理解能力自然决定处理需求，而非依赖外部压缩规则。在六个问答数据集（最长达40K tokens）上，使用三种模型家族（LLaMA/Qwen/Mistral，1B-70B参数）的实验表明，该方法平均实现1.33倍的token减少，同时准确率提升3.4%。此外，在相同token减少率下，其表现优于其他上下文效率方法。还观察到一种涌现现象：小模型需显式探测充分性，而大模型可通过提示（prompting）实现内在自我评估。
原文摘要 · Abstract (English)
Large language models (LLMs) process entire input contexts indiscriminately, which is inefficient when the information required to answer a query is localized within the context. We present dynamic context cutoff, a novel method enabling LLMs to self-terminate processing upon acquiring sufficient task-relevant information. Through analysis of model internals, we discover that specific attention heads inherently encode "sufficiency signals" -- detectable through lightweight classifiers -- that predict when critical information has been processed. This reveals a new efficiency paradigm: models' internal understanding naturally dictates processing needs rather than external compression heuristics. Comprehensive experiments across six QA datasets (up to 40K tokens) with three model families (LLaMA/Qwen/Mistral, 1B-70B) demonstrate 3.4% accuracy improvement while achieving 1.33x token reduction on average. Furthermore, our method demonstrates superior performance compared to other context efficiency methods at equivalent token reduction rates. Additionally, we observe an emergent scaling phenomenon: while smaller models require probing for sufficiency detection, larger models exhibit intrinsic self-assessment capabilities through prompting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。