用小模型压缩大模型输入，提升效率并精准控制成本。
Cmprsr: Abstractive Token-Level Question-Agnostic Prompt Compressor
- 用小模型压缩大模型的输入提示，降低使用成本。
- 在多个数据集上优于现有抽取式和原始摘要式方法，且压缩率可控。
- 适用于长文本和短任务，适合需要灵活控制成本的研究者。
为降低黑盒大语言模型（LLM）的使用成本，我们提出一种新型提示压缩范式：使用小型LLM压缩大模型的输入。我们构建了首个涵盖25个开源与闭源模型的全面基准测试，揭示各模型在保留语义重要信息及遵循用户指定压缩率（CR）方面存在显著差异。通过Textgrad优化元提示，进一步提升了gpt-4.1-mini的性能。我们识别出表现最佳的开源模型Qwen3-4B，结合监督微调（SFT）与组相对策略优化（GRPO）进行后训练，同时优化压缩率遵从性与下游任务表现。最终模型命名为Cmprsr，其在MeetingBank、LongBench等长文本及GSM8k等短提示数据集上，均优于提取式与原始抽象式压缩方法，且在全压缩率范围内保持高精度压缩率控制，展现跨长度与领域的泛化能力。
原文摘要 · Abstract (English)
Motivated by the high costs of using black-box Large Language Models (LLMs), we introduce a novel prompt compression paradigm, under which we use smaller LLMs to compress inputs for the larger ones. We present the first comprehensive LLM-as-a-compressor benchmark spanning 25 open- and closed-source models, which reveals significant disparity in models' compression ability in terms of (i) preserving semantically important information (ii) following the user-provided compression rate (CR). We further improve the performance of gpt-4.1-mini, the best overall vanilla compressor, with Textgrad-based compression meta-prompt optimization. We also identify the most promising open-source vanilla LLM - Qwen3-4B - and post-train it with a combination of supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO), pursuing the dual objective of CR adherence and maximizing the downstream task performance. We call the resulting model Cmprsr and demonstrate its superiority over both extractive and vanilla abstractive compression across the entire range of compression rates on lengthy inputs from MeetingBank and LongBench as well as short prompts from GSM8k. The latter highlights Cmprsr's generalizability across varying input lengths and domains. Moreover, Cmprsr closely follows the requested compression rate, offering fine control over the cost-quality trade-off.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。