arXiv:2606.24083cs.CLcs.AI2026-06

研究大模型在输入输出压缩下的表现,发现输出压缩省钱,输入压缩反而更贵。

CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression

论文配图:CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression
图 1 · 摘自论文原文
  • 设计双通道评估框架,同时测试输入和输出压缩对成本与准确率的影响。
  • 输出压缩可节省1.4-3倍推理成本,输入压缩使总成本上升1.15-2.7倍。
  • 即使答案正确,压缩后文本仍与原始输出差异显著,影响可靠性。

‘说短点,丢语法,省令牌’这一原始风格被广泛推崇以降低推理成本,但实际是否节流取决于压缩的是用户提示还是模型回复。我们提出Cavewoman,一种双通道评估协议,在五个数据集上对八种模型在五种压缩等级下进行评估,同时测量任务准确率、实际每项成本及与模型无约束参考文本的一致性。结果显示,输出压缩在多数API模型(每模型节省1.4-2.4倍,最优达3倍)和所有四款开源模型(按公共定价)中有效降低实际成本;而输入压缩则带来严格意义上的双输:净成本上升(五基准平均约1.15倍,最差数据集达1.8倍,强压缩下高达2.7倍),因模型为弥补准确率下降而生成更长回复。此外,表面文本与无约束基线的语义一致性明显下降:非推理类模型中,约一半生成结果虽正确,但其表层文本不再蕴含模型自身无约束生成的结果。该偏差在长度控制重评分、多重比较校正及互补语义度量下依然存在。代码与数据见https://github.com/danielle34/cavewoman。

原文摘要 · Abstract (English)

"Talk short. Drop grammar. Save token." This caveman style is widely promoted as a way to cut inference cost, but whether it actually saves anything depends on which channel (the user's prompt or the model's response) is being compressed. We present Cavewoman, a two-channel evaluation protocol that scores every generation on task accuracy, realized per-item cost, and reference-text agreement against the model's unconstrained reference. We evaluate eight models on five datasets at five reduction levels, with both channels measured on the same items. Output compression cuts realized cost on most API models (1.4-2.4x per model, up to 3x in the best case) and on all four open-weight models under public-tier pricing. Input compression has the opposite effect, a strict lose-lose: it raises net cost rather than lowering it (~1.15x on the five-benchmark mean, up to 1.8x on the worst dataset and 2.7x under stronger compression), because models compensate with longer responses even as accuracy collapses. Under the same setting, surface text diverges from the unconstrained reference: on the non-reasoning models, roughly half of all generations are correct yet their surface text no longer entails the model's own unconstrained baseline generation. The divergence survives length-controlled re-scoring, multiple-comparisons correction, and replication under complementary semantic measures. Code and data are available at https://github.com/danielle34/cavewoman.

大模型推理优化压缩评估语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。