研究六种提示压缩方法，降低大模型推理成本。
An Empirical Study on Prompt Compression for Large Language Models
- 对比六种提示压缩方法，评估其对模型表现的影响。
- 长上下文任务中压缩效果更显著，适度压缩能提升性能。
- 适合关注大模型推理效率与成本优化的研究者。
提示工程使大语言模型（LLMs）能够执行多种任务。然而，过长的提示会显著增加计算复杂度和经济成本。为解决此问题，本文系统研究了六种针对大语言模型的提示压缩方法，旨在减少提示长度的同时保持模型响应质量。我们进行了全面分析，涵盖生成性能、模型幻觉、多模态任务有效性、词语遗漏分析等多个方面。在13个数据集上进行评估，包括新闻、科学文章、常识问答、数学问答、长上下文问答和视觉问答（VQA）等。实验结果表明，提示压缩在长上下文任务中对模型性能影响更大；在Longbench评测中，适度压缩甚至能提升模型表现。相关代码与数据已公开于https://github.com/3DAgentWorld/Toolkit-for-Prompt-Compression。
原文摘要 · Abstract (English)
Prompt engineering enables Large Language Models (LLMs) to perform a variety of tasks. However, lengthy prompts significantly increase computational complexity and economic costs. To address this issue, we study six prompt compression methods for LLMs, aiming to reduce prompt length while maintaining LLM response quality. In this paper, we present a comprehensive analysis covering aspects such as generation performance, model hallucinations, efficacy in multimodal tasks, word omission analysis, and more. We evaluate these methods across 13 datasets, including news, scientific articles, commonsense QA, math QA, long-context QA, and VQA datasets. Our experiments reveal that prompt compression has a greater impact on LLM performance in long contexts compared to short ones. In the Longbench evaluation, moderate compression even enhances LLM performance. Our code and data is available at https://github.com/3DAgentWorld/Toolkit-for-Prompt-Compression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。