用多重图剪枝压缩提示词，让重要信息不丢失。
Mapping Text to Multiplex Graph: Prompt Compression as Lévy Walk-Guided Graph Pruning
- 将文本转为多重图，同时建模细粒度依赖和粗粒度语义关系。
- 在4倍压缩比下，长文本任务平均得分49.3，优于现有方法。
- 适合需要高效压缩长文本的LLM应用，如问答与摘要。
现有提示词压缩方法将文本视为扁平的词元序列，无法捕捉关键信息分布于多处且通过局部语法依赖与全局语义关系相连的特性。此类关系结构天然适合用图表示，其中词元或句子为节点，依赖关系为边。为此，我们提出RAGP，将提示词压缩建模为在多重图上的冗余感知图剪枝，该图联合建模细粒度注意力依赖与粗粒度语义关系。为高效识别此异构结构（密集局部子图与稀疏全局连接）中的非冗余节点，我们采用重尾步长分布的莱维游走，自然平衡局部探索与全局寻访。在LongBench上的实验表明，RAGP在4倍压缩比下平均得分为49.3，优于基于LLM的方法LongLLMLingua（3倍压缩比下得分为48.8）。此外，RAGP还在多个任务上超越当前最优视觉基文本压缩范式。代码已公开于https://anonymous.4open.science/r/RAGP-B0CB。
原文摘要 · Abstract (English)
Existing prompt compression methods treat text as flat token sequences, failing to capture the distributed nature of important information, which is often spread across multiple locations and connected through both local syntactic dependencies and global semantic relations. Such relational structure is naturally represented as a graph, where tokens or sentences become nodes and their dependencies become edges. To this end, we propose RAGP, which formulates prompt compression as Redundancy-Aware Graph Pruning on a multiplex graph that jointly models fine-grained attention-based dependencies and coarse-grained semantic relations. To efficiently identify non-redundant nodes in this heterogeneous structure (dense local subgraphs and sparse global connections), we employ Levy walks whose heavy-tailed step distribution naturally balances local exploitation with global exploration. Experiments on LongBench show that RAGP achieves an average score of 49.3 under a 4x compression ratio, outperforming existing LLM-based compression methods, such as LongLLMLingua, which attains 48.8 at a 3x compression ratio. Besides, RAGP also surpasses state-of-the-art vision-based text compression paradigms on multiple tasks. The code is available at https://anonymous.4open.science/r/RAGP-B0CB.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。