通过查询引导的激活补全，高效处理长文本信息检索。
Boosting Long-Context Management via Query-Guided Activation Refilling
- 构建双层缓存结构，分层存储全局与局部信息。
- 查询动态触发局部信息补全，提升答案准确性。
- 适合需要长上下文理解的问答与信息抽取任务。
长上下文处理对大语言模型（LLMs）构成挑战，源于其固有的上下文窗口限制及大量键值（KV）激活带来的计算负担，严重影响效率。在信息检索任务中，完整感知上下文往往非必要，因查询的信息需求可随复杂度动态变化，从局部细节到全局视角不等。然而，现有方法难以有效适应这种动态需求。本文提出一种基于查询引导的激活补全（ACRE）方法，用于长上下文信息检索任务。ACRE 构建双层 KV 缓存：第一层（L1）紧凑捕捉全局信息，第二层（L2）提供详细局部信息。两层间建立代理关系，使输入查询可访问 L1 并动态从 L2 补充相关条目。该机制融合全局理解与查询特定局部细节，从而提升答案解码效果。在多个长上下文信息检索数据集上的实验表明，ACRE 在性能和效率上均有显著提升。
原文摘要 · Abstract (English)
Processing long contexts poses a significant challenge for large language models (LLMs) due to their inherent context-window limitations and the computational burden of extensive key-value (KV) activations, which severely impact efficiency. For information-seeking tasks, full context perception is often unnecessary, as a query's information needs can dynamically range from localized details to a global perspective, depending on its complexity. However, existing methods struggle to adapt effectively to these dynamic information needs. In the paper, we propose a method for processing long-context information-seeking tasks via query-guided Activation Refilling (ACRE). ACRE constructs a Bi-layer KV Cache for long contexts, where the layer-1 (L1) cache compactly captures global information, and the layer-2 (L2) cache provides detailed and localized information. ACRE establishes a proxying relationship between the two caches, allowing the input query to attend to the L1 cache and dynamically refill it with relevant entries from the L2 cache. This mechanism integrates global understanding with query-specific local details, thus improving answer decoding. Experiments on a variety of long-context information-seeking datasets demonstrate ACRE's effectiveness, achieving improvements in both performance and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。