动态调整上下文窗口，让模型在长文本生成时更省资源且不丢质量。
UT-ACA: Uncertainty-Triggered Adaptive Context Allocation for Long-Context Inference
- 根据每个词的不确定度实时调整上下文长度，避免固定预算浪费。
- 实验显示平均上下文使用量显著降低，生成质量保持不变。
- 适合需要高效处理长文档的场景，如法律、医疗等专业领域。
长上下文推理对大语言模型仍是挑战，源于注意力稀释和分布外性能下降。上下文选择通过仅关注关键键值缓存条目缓解此问题，但多数方法在解码过程中采用固定上下文预算，而未考虑不同词元的上下文需求差异。为此，我们提出不确定性触发的自适应上下文分配（UT-ACA），一种推理阶段框架，可根据词元级不确定性动态调整上下文窗口。UT-ACA学习一个不确定性检测器，结合语义嵌入与基于逻辑置信度的信息，并考虑解码步骤间的不确定性累积。当证据不足时，系统会部分回滚、扩大上下文窗口并重新生成该词元以获得额外支持。实验表明，UT-ACA在长上下文设置下显著降低平均上下文使用量，同时保持生成质量。代码与数据集见：https://github.com/Tommy307/UT-ACA。
原文摘要 · Abstract (English)
Long-context inference remains challenging for large language models due to attention dilution and out-of-distribution degradation. Context selection mitigates this limitation by attending to a subset of key-value cache entries, yet most methods allocate a fixed context budget throughout decoding despite highly non-uniform token-level contextual demands. To address this issue, we propose Uncertainty-Triggered Adaptive Context Allocation (UT-ACA), an inference-time framework that dynamically adjusts the context window based on token-wise uncertainty. UT-ACA learns an uncertainty detector that combines semantic embeddings with logit-based confidence while accounting for uncertainty accumulation across decoding steps. When insufficient evidence is indicated, UT-ACA selectively rolls back, expands the context window, and regenerates the token with additional support. Experiments show that UT-ACA substantially reduces average context usage while preserving generation quality in long-context settings. Code and dataset are available at https://github.com/Tommy307/UT-ACA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。