用语义图量化信息熵,压缩长文本保留核心含义。
Concept than Document: Context Compression via AMR-based Conceptual Entropy
- 构建文本的抽象语义图,按节点信息熵筛选关键概念。
- 在PopQA和EntityQuestions上准确率提升,上下文长度减少40%以上。
- 适合需要高效长文本处理的问答与推理场景。
大型语言模型在处理长上下文时面临信息过载,尤其在检索增强生成(RAG)中,大量支持文档常引入冗余内容,不仅降低推理准确性,还增加计算开销。本文提出一种无监督的上下文压缩框架,利用抽象语义表示(AMR)图保留语义关键信息并过滤无关文本。通过量化AMR图中各节点的语义熵,估计节点的概念重要性,从而保留核心语义。具体而言,从原始上下文构建AMR图,计算每个节点的概念熵,并筛选出具有显著信息量的节点,形成更紧凑且语义聚焦的上下文。在PopQA和EntityQuestions数据集上的实验表明,该方法优于原始输入和其他基线模型,在显著缩短上下文长度的同时实现更高准确率。据我们所知,这是首个将基于AMR的概念熵用于上下文压缩的工作,展示了稳定语言特征在上下文工程中的潜力。
原文摘要 · Abstract (English)
Large Language Models (LLMs) face information overload when handling long contexts, particularly in Retrieval-Augmented Generation (RAG) where extensive supporting documents often introduce redundant content. This issue not only weakens reasoning accuracy but also increases computational overhead. We propose an unsupervised context compression framework that exploits Abstract Meaning Representation (AMR) graphs to preserve semantically essential information while filtering out irrelevant text. By quantifying node-level entropy within AMR graphs, our method estimates the conceptual importance of each node, enabling the retention of core semantics. Specifically, we construct AMR graphs from raw contexts, compute the conceptual entropy of each node, and screen significant informative nodes to form a condensed and semantically focused context than raw documents. Experiments on the PopQA and EntityQuestions datasets show that our method outperforms vanilla and other baselines, achieving higher accuracy while substantially reducing context length. To the best of our knowledge, this is the first work introducing AMR-based conceptual entropy for context compression, demonstrating the potential of stable linguistic features in context engineering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。