发现大模型重复惩罚机制存在缺陷，导致输出混乱且依赖模型设置。
Gauge dependence and structured-output corruption in sign-branched repetition penalties: measurements across models, inference stacks, and alternative repetition controls
- 用归一化概率替代原始logits做惩罚，可消除签名分支带来的偏差
- 原方法在theta=1.3时使97%的合规输出降至23%
- 该问题影响主流推理引擎，修复方案已可用
当前大模型推理中广泛采用的乘法重复惩罚机制，根据每个logit的正负号进行分支处理（正数除以theta，负数乘以theta）。但由于softmax对所有logit加常数不变，模型logit的零点是任意的，该符号分支实际上依赖于一个训练目标未约束的量。这导致两个可测量后果：(1) 惩罚本身无明确定义——将logits整体平移常数在theta=1时为无操作，但在常规theta=1.3下却改变58%-96%的贪婪采样结果；不同检查点零点差异大，固定rep_penalty在不同模型上实际执行不同操作。(2) 严重破坏结构化输出：在200个真实世界JSON模式上，theta=1.3使合规输出率从97%暴跌至23%。我们验证了将惩罚应用于归一化后的概率（而非原始logits）可完全消除上述问题。HuggingFace已提供LogitNormalization算子，但默认关闭且位于惩罚之后。本文报告了机制、五种模型（最大7B，含基础与强化学习微调）在WikiText-103前缀上的测试，以及两组代码模型在HumanEval和JSONSchemaBench上的复现结果，并证实问题存在于vLLM和llama.cpp内部采样器中。
原文摘要 · Abstract (English)
The multiplicative repetition penalty shipped across the LLM inference ecosystem (HuggingFace, vLLM, llama$.$cpp, and a dozen further engines) branches on the sign of each raw logit (divide positives by theta, multiply negatives). But the softmax is unchanged by adding a constant to every logit, so a model's logit zero-point is arbitrary, and the sign-branch reads that arbitrary point. The sign-branch is itself the accepted fix for an earlier bug, so the accepted fix branches on a quantity the training objective leaves unconstrained. Two measurable consequences follow. (1) The penalty is not well-defined: re-centring a model's logits by a constant is a provable no-op at theta=1, yet at a routine theta=1.3 it changes 58-96% of greedy tokens, where subtractive and normalized penalties change none; real checkpoints sit at widely different zero-points, so a fixed repetition_penalty is a different operation on every model. (2) It corrupts structured output: on 200 real-world JSON schemas, theta=1.3 drops the rate of valid, schema-conformant output from 97% to 23%. In our measurements, applying the penalty to normalized log-probabilities instead of raw logits removes both effects. HuggingFace already ships that operator (LogitNormalization); today it is off by default and applied after the penalty. This note gives the mechanism, the measurements (five models up to 7B, base and RLHF, on WikiText-103 prefixes; two code models on HumanEval and JSONSchemaBench; both effects replicated inside vLLM and llama$.$cpp through their own samplers on the same inputs), and the normalized variant.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。