用分块重写方法压缩提示词，降低大模型推理成本。
SCOPE: A Generative Approach for LLM Prompt Compression
- 将提示词分块后重写为更简洁的版本，保持语义连贯。
- 在高压缩比下仍保持生成质量，优于现有方法。
- 无需训练，适合对推理效率有要求的应用场景。
现代大模型应用常输入长上下文，导致推理成本高、延迟大，甚至超出上下文限制。提示词压缩通过减少输入长度同时最小化生成质量损失来解决该问题。本文提出SCOPE，一种基于分块重写的无训练生成式压缩框架。不同于传统删减令牌的方法，SCOPE将提示词划分为语义连贯的块，并对每块进行精炼重写，再重构为有意义的紧凑提示。我们设计了多项优化技术，有效保留关键信息与文本连贯性，并支持细粒度压缩率控制。在问答和摘要等典型任务上进行了广泛评估，结果表明，在多数设置下SCOPE均优于对比的选取式压缩基线，尤其在高压缩比时表现更优。
原文摘要 · Abstract (English)
A big issue in modern LLM applications is they tend to feed long context to LLM, which results in high inference cost and latency, and may exceed the context limit. Prompt compression addresses this issue by reducing the length of input context with minimum loss of generation quality, i.e, the goal of prompt compression is to shorten the LLM input while maintaining a high generation quality. To overcome these limitations, we propose SCOPE, a training-free generative prompt compression framework based on chunk-level rewriting. Unlike the existing token removal methods, our method centers at a chunking-and-summarization mechanism. Specifically, SCOPE splits a prompt into semantically coherent chunks and rewrites the chunks to be more concise. Then the chunks are reconstructed into a meaningful prompt. Additionally, we design several optimization techniques for SCOPE, effectively preserving critical information and text coherence in compression, as well as providing finer-grained control of the compression ratio. We conduct extensive evaluation on typical LLM applications like question-answering and summarization. Results show that SCOPE consistently outperforms the evaluated selective compression baselines across most settings, with particularly strong gains at high compression ratios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。