用基本话语单元重构文本结构,实现高效低失真的长文本压缩。
From Context to EDUs: Faithful and Structured Context Compression via Elementary Discourse Unit Decomposition
- 将文本拆解为带源位置锚定的语义单元,避免幻觉。
- 在248篇文档上结构预测准确率领先,推理成本降低超50%。
- 适合长文档问答与深度搜索等需要结构保持的场景。
长文本处理仍是大语言模型的关键瓶颈,尤其在长文档问答和自主代理任务中,长输入带来高计算开销和噪声。现有压缩方法常通过删减离散分词破坏局部连贯性,或依赖隐式编码导致位置偏差,且不兼容闭源API。为此,我们提出基于基本话语单元(EDU)的显式压缩框架——EDU-based Context Compressor。该方法将压缩重构为‘结构化→选择’两阶段:首先,LingoEDU将线性文本转为严格锚定源位置的EDU关系树,杜绝幻觉;其次,轻量级排序模块选取与查询相关子树并线性化输出。为评估结构理解能力,我们发布人工标注的StructBench数据集(248篇多样化文档)。实验证明,本方法在结构预测上达到当前最优性能,显著优于前沿大模型,同时大幅降低计算成本。此外,结构感知压缩在长上下文任务与复杂深度搜索场景中均大幅提升下游表现。
原文摘要 · Abstract (English)
Managing extensive context remains a critical bottleneck for Large Language Models (LLMs), particularly in applications like long-document question answering and autonomous agents where lengthy inputs incur high computational costs and introduce noise. Existing compression techniques often disrupt local coherence through discrete token removal or rely on implicit latent encoding that suffers from positional bias and incompatibility with closed-source APIs. To address these limitations, we introduce the EDU-based Context Compressor, a novel explicit compression framework designed to preserve both global structure and fine-grained details. Our approach reformulates context compression as a structure-then-select process. First, our LingoEDU transforms linear text into a structural relation tree of Elementary Discourse Units (EDUs) which are anchored strictly to source indices to eliminate hallucination. Second, a lightweight ranking module selects query-relevant sub-trees for linearization. To rigorously evaluate structural understanding, we release StructBench, a manually annotated dataset of 248 diverse documents. Empirical results demonstrate that our method achieves state-of-the-art structural prediction accuracy and significantly outperforms frontier LLMs while reducing costs. Furthermore, our structure-aware compression substantially enhances performance across downstream tasks ranging from long-context tasks to complex Deep Search scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。