提出新压缩框架，让大模型更高效处理长文本。
Fix the Structural Bottleneck: Context Compression via Explicit Information Transmission
- 通过显式信息传输机制，优化压缩令牌间协作与信号传递
- 在12个数据集上平均提升F1达18.5%，压缩速度超基线2倍以上
- 仅增加1%可训练参数，适合资源受限场景下的长文本应用
长上下文大模型代理常面临令牌数、内存和延迟成本上升的问题，高效上下文压缩对实际部署至关重要。现有基于大模型的压缩方法仍显著逊于使用完整上下文的表现。我们发现该差距部分源于其无法有效保留上下文信息。本文从结构视角重新审视压缩问题，识别出标准大模型压缩器的两大瓶颈：压缩令牌在信息聚合中协调不足，以及层间信号稀释导致中间隐藏状态有用信号减弱。为此，我们提出ComprExIT框架，通过显式信息传输实现改进：自适应选择冻结大模型各层特征，并通过全局协同传输计划将锚点信息分配至压缩槽位。在12个数据集上的实验表明，ComprExIT持续优于强基线，平均F1提升最高达18.5%，仅增加约1%可训练参数，且压缩速度比最快基线快2倍以上。代码将在论文录用后公开。
原文摘要 · Abstract (English)
Long-context LLM agents often struggle with growing token, memory, and latency costs, making efficient context compression essential for practical deployment. Existing LLM-as-a-compressor methods remain noticeably inferior to using the full context. We find that this gap partly stems from their inability to preserve contextual information effectively. In this work, we revisit context compression from a structural perspective and identify two key bottlenecks in standard LLM-based compressors: limited coordination among compression tokens during information aggregation, and layerwise dilution that weakens useful signals from intermediate hidden states. To address these limitations, we propose ComprExIT, a new context compression framework based on explicit information transmission. ComprExIT adaptively selects features across frozen LLM layers, then allocates information from anchors to compression slots through a globally coordinated transport plan. Experiments on 12 datasets show that ComprExIT consistently outperforms strong soft-compression baselines, improving average F1 by up to 18.5%, while adding only ~1% trainable parameters and achieving more than 2x faster compression than the fastest baselines. The code will be released upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。