无需训练的分层提示压缩方法，提升长文本理解效率与准确性
BEAVER: A Training-Free Hierarchical Prompt Compression Method via Structure-Aware Page Selection
- 通过结构感知的页面选择实现分层压缩，避免传统删减导致的信息碎片化
- 在128k上下文上降低26.4倍延迟，多针检索任务中保持高精度
- 适合需要高吞吐、低延迟的长文档处理场景，如智能客服与法律分析
大模型上下文窗口的指数级扩展虽提升了长文档理解能力，却带来了严重的推理延迟和信息利用瓶颈。现有压缩方法常因高训练成本或激进的令牌删减导致语义断裂。本文提出BEAVER，一种无需训练的新型框架，将压缩从线性删除转向结构感知的分层选择。通过双路径池化将可变长度上下文映射为密集的页级张量，最大化硬件并行性；结合语义与词汇双分支选择及句子平滑机制，保障语篇完整性。在四个长上下文基准上的广泛评估表明，BEAVER性能媲美顶尖方法LongLLMLingua。尤其在RULER基准上，其多针检索任务保持高保真度，而基线方法性能显著下降。效率方面，在128k上下文上延迟降低26.4倍，为高吞吐应用提供可扩展解决方案。代码已开源：https://cslikai.cn/BEAVER/
原文摘要 · Abstract (English)
The exponential expansion of context windows in LLMs has unlocked capabilities for long-document understanding but introduced severe bottlenecks in inference latency and information utilization. Existing compression methods often suffer from high training costs or semantic fragmentation due to aggressive token pruning. In this paper, we propose BEAVER, a novel training-free framework that shifts compression from linear token removal to structure-aware hierarchical selection. BEAVER maximizes hardware parallelism by mapping variable-length contexts into dense page-level tensors via dual-path pooling, and preserves discourse integrity through a hybrid planner combining semantic and lexical dual-branch selection with sentence smoothing. Extensive evaluations on four long-context benchmarks demonstrate that BEAVER achieves comparable performance to state-of-the-art (SOTA) methods like LongLLMLingua. Notably, on the RULER benchmark, BEAVER maintains high fidelity in multi-needle retrieval where baselines deteriorate. Regarding efficiency, BEAVER reduces latency by 26.4x on 128k contexts, offering a scalable solution for high-throughput applications. Our code is available at https://cslikai.cn/BEAVER/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。