用阅读理解思路压缩提示词，让大模型更省力、更准。
From Reading to Compressing: Exploring the Multi-document Reader for Prompt Compression
- 用FiD架构分析提示词，自动找关键内容
- 压缩后提示词长度减少80%，跨领域性能提升6%
- 无需伪标签训练，适合真实场景部署
大语言模型在各种任务中通过先进提示技术取得了显著性能提升，但提示词过长导致计算成本高且关键信息被掩盖。提示压缩被提出以缓解此问题，但仍面临（i）捕捉全局上下文和（ii）有效训练压缩器的挑战。为此，我们提出一种新型提示压缩方法——阅读到压缩（R2C），采用融合解码器（FiD）架构识别提示中的重要信息。具体而言，利用FiD的交叉注意力分数判断提示中重要的段落和句子。R2C在不损害语义一致性的情况下有效捕捉全局上下文，同时避免了训练压缩器所需的伪标签。实验证明，R2C能保留关键上下文，在跨领域评估中使大模型性能提升6%，同时将提示长度减少80%。
原文摘要 · Abstract (English)
Large language models (LLMs) have achieved significant performance gains using advanced prompting techniques over various tasks. However, the increasing length of prompts leads to high computational costs and often obscures crucial information. Prompt compression has been proposed to alleviate these issues, but it faces challenges in (i) capturing the global context and (ii) training the compressor effectively. To tackle these challenges, we introduce a novel prompt compression method, namely Reading To Compressing (R2C), utilizing the Fusion-in-Decoder (FiD) architecture to identify the important information in the prompt. Specifically, the cross-attention scores of the FiD are used to discern essential chunks and sentences from the prompt. R2C effectively captures the global context without compromising semantic consistency while detouring the necessity of pseudo-labels for training the compressor. Empirical results show that R2C retains key contexts, enhancing the LLM performance by 6% in out-of-domain evaluations while reducing the prompt length by 80%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。