用压缩激活签名提升大模型长文本理解能力
MiA-Signature: Approximating Global Activation for Long-Context Understanding
- 通过子模函数筛选关键概念，生成全局激活的紧凑表示
- 在多个长上下文任务中，相比基线提升性能且计算开销可控
- 适合需要高效处理长文档的检索增强与智能体系统
认知科学研究表明，可报告的意识访问与分布式记忆系统中的全局点火相关，但个体无法直接访问或枚举所有激活内容。这提示认知可能依赖一种紧凑表征来近似激活对下游处理的全局影响。受此启发，我们提出「心智景观激活签名（MiA-Signature）」，即查询引发的全局激活模式的压缩表示。在大语言模型中，该表示通过基于子模性的高阶概念选择实现，覆盖激活的上下文空间，并可选地通过轻量级工作记忆迭代更新优化。所得的 MiA-Signature 作为条件信号，近似完整激活状态的影响，同时保持计算可行性。将 MiA-Signature 集成到 RAG 和智能体系统中，在多个长上下文理解任务上均获得一致性能提升。
原文摘要 · Abstract (English)
A growing body of work in cognitive science suggests that reportable conscious access is associated with \emph{global ignition} over distributed memory systems, while such activation is only partially accessible as individuals cannot directly access or enumerate all activated contents. This tension suggests a plausible mechanism that cognition may rely on a compact representation that approximates the global influence of activation on downstream processing. Inspired by this idea, we introduce the concept of \textbf{Mindscape Activation Signature (MiA-Signature)}, a compressed representation of the global activation pattern induced by a query. In LLM systems, this is instantiated via submodular-based selection of high-level concepts that cover the activated context space, optionally refined through lightweight iterative updates using working memory. The resulting MiA-Signature serves as a conditioning signal that approximates the effect of the full activation state while remaining computationally tractable. Integrating MiA-Signatures into both RAG and agentic systems yields consistent performance gains across multiple long-context understanding tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。