发现大模型推理时大量浪费计算资源在无意义重复上，提出轻量级方法实时清除
Word Salad Chopper: Reasoning Models Waste A Ton Of Decoding Budget On Useless Repetitions, Self-Knowingly
- 用单层线性分类器实时识别推理中的无意义重复模式
- 清除重复后可节省40%以上输出长度，质量损失极小
- 无需改动主模型，适合所有需优化用户体验的大型推理应用
大型推理模型常因输出令牌成本过高而受制约。我们发现其中大量令牌为无价值的自我重复——即‘文字沙拉’——消耗解码预算却无实际贡献。有趣的是，模型在陷入此类循环时表现出自知：每个推理片段末尾的<\n\n>令牌隐藏状态具有可被检测的模式。通过一个单层线性分类器可实时识别该行为。一旦检测到，仅需附加简单裁剪与再生提示，即可实现显著的长度压缩，且质量损失微乎其微。本文提出WordSaladChopper（WSC）——一种轻量、即插即用的组件，仅移除语义冗余令牌，对原推理轨迹影响极小。因其低开销、强收益，且文字沙拉本无语义价值，我们认为WSC或类似模块应成为所有注重用户体验的大型推理模型的标配。代码已开源：https://github.com/wenyaxie023/WordSaladChopper。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) are often bottlenecked by the high cost of output tokens. We show that a significant portion of these tokens are useless self-repetitions - what we call "word salad" - that exhaust the decoding budget without adding value. Interestingly, we observe that LRMs are self-aware when trapped in these loops: the hidden states of <\n\n> tokens trailing each reasoning chunk exhibit patterns that allow us to detect word salad behavior on-the-fly via a single-layer linear classifier. Once detected, a simple chop appended by a straightforward regeneration prompt yields substantial length savings with minimal quality loss. Our work offers WordSaladChopper (WSC) - a lightweight, turnkey component for LRM that is minimally invasive to its reasoning trajectory by only removing semantically redundant tokens. Given its low overhead, strong savings, and the lack of semantic value of word salad tokens, we believe it is not too far-fetched to argue that WSC - or a similar component - is a must-have for all LRM applications with user experience in mind. Our code is publicly available at https://github.com/wenyaxie023/WordSaladChopper.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。