让大模型压缩上下文更可靠，按性能底线自动找最优压缩比。
PoC: Performance-oriented Context Compression for Large Language Models via Performance Prediction
- 不指定压缩比例，而是设定最低性能要求，由预测器自动找最佳压缩度。
- 上下文感知的预测器误差更低，使压缩后模型性能更稳定。
- 适合追求高效且稳定推理的工业部署场景。
尽管上下文压缩能通过缩短输入来缓解大语言模型的推理开销，但现有方法需预先设定压缩比例或长度，易导致性能不可预测地下降，阻碍其可靠部署。本文提出性能导向的上下文压缩（PoC）范式，开发者只需设定可接受的性能下限，而非压缩比。PoC利用轻量级性能预测器，在调用现成压缩器前自动寻找满足该约束的最激进压缩比。我们设计并对比了两种预测器：一种与上下文无关的简单版本，另一种考虑输入内在可压缩性的上下文感知版本。在问答和摘要两个基准上，上下文感知预测器始终比无关版本具有更低的预测误差，相应地，上下文感知的PoC实现了更优的整体性能。本工作为大模型上下文压缩的可靠、高效、性能感知部署开辟了新路径。
原文摘要 · Abstract (English)
While context compression can mitigate the growing inference costs of Large Language Models (LLMs) by shortening contexts, existing methods that specify a target compression ratio or length suffer from unpredictable performance degradation, hindering their reliable deployment. We introduce a paradigm shift to Performance-oriented Context Compression (PoC), where developers specify an acceptable performance floor instead of a compression ratio. PoC employs a lightweight performance predictor to automatically find the most aggressive compression ratio that satisfies this constraint before steering an off-the-shelf compressor. We design and compare two predictor variants: a simple context-agnostic predictor and a more sophisticated context-aware one that considers the input's inherent compressibility. On both question-answering and summarization benchmarks, the context-aware predictor consistently achieves lower performance prediction error than the context-agnostic predictor, while the resulting context-aware PoC attains a superior overall performance. Our work paves the way for a more reliable, efficient, and performance-aware deployment of context compression for LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。