用语言学规则压缩提示词,省去模型计算,降本增效。
Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressors

- 基于语言学规则构建无模型依赖的压缩方法
- 轻中度压缩下性能接近先进策略,且无需推理计算
- 适合追求低成本部署的NLP应用开发者
提示词压缩可降低大模型推理成本,但现有方法依赖语言模型前向传播评估词元重要性,代价高昂。语言学研究长期关注信息识别,其可操作化的线索可转化为确定性规则。本文提出仅用语言学规则作为压缩器,在部署时无需任何语言模型前向传播,仅通过CPU处理完成压缩。采用双路径评估协议平衡压缩质量与重建保真度。在短文本、多文档推理和对话记忆问答数据集上,演化出的语言学压缩器表现媲美最新先进策略。轻中度压缩时效果最佳,过度压缩则性能下降;直接路径与重建路径呈现不同趋势。演化分析表明,高效压缩融合多层级语言信号,压缩率升高时规则从删词转向句子提取。
原文摘要 · Abstract (English)
Prompt compression shortens LLM input to reduce inference cost, yet existing methods score token importance through LM forward passes. It remains questionable whether such nuanced, costly token selection is necessary. Compression requires identifying informative content, a problem that linguistic research has long addressed through cues that can be operationalized as deterministic rules. We therefore ask: can \textbf{linguistic rules alone} serve as effective prompt compressors, without LM-based scoring at compression time? To address this, we conduct offline evolutionary search over lexical, syntactic, semantic, and discourse seeds to find competitive rule combinations. The resulting linguistic compressor requires no LM forward pass at deployment and uses only CPU-side processing for compression. We evaluate it with a dual-path protocol to balance compression quality and reconstruction fidelity. Across short passages, multi-document reasoning, and dialogue-memory QA datasets, evolved compressors achieve performance similar to that of recent advanced prompt-compression strategies. Performance is strongest under light-to-moderate compression and degrades as compression becomes more aggressive, while the Direct and Reconstruction paths exhibit distinct patterns. Evolutionary analysis reveals that effective compression fuses signals across linguistic levels and, as the compression ratio increases, rules shift from token pruning to sentence extraction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。