用贝叶斯思想修复自回归模型的重复生成问题
Bayesian Repetition Penalty: A Principled Adjacent-Conditional Framework for Reversing Attention Collapse in Autoregressive Language Models
- 通过相邻条件概率比较预测频率与先验，识别异常自信
- 使2-gram重复率从0.073降至接近0，保持生成质量
- 无需修改训练流程，可作为后处理修复已崩溃模型
自回归语言模型中的注意力坍缩问题——表现为模型陷入自我强化的重复词元循环——是长期存在的缺陷，现有解码阶段启发式方法无法根治。本文提出一种基于贝叶斯原理的邻接条件框架，通过比较词元的实际出现频率与其语料库先验，检测由坍缩生成模式引发的异常置信度。提出的自归一化惩罚比值 $R=f(m,n,p)/f(np,n,p)$ 无需人为标准化，且具有零近似误差的闭式对数偏移。该修正量与损失梯度解耦，通过指数移动平均累积为冻结输出层偏置，可作为无需侵入标准训练流程的修复机制部署。在15亿参数模型上的实验表明，该冻结偏置机制能有效挽救已陷入坍缩吸引子的模型，将2-gram重复率从0.073降至接近0，同时维持生成质量。
原文摘要 · Abstract (English)
Attention collapse in autoregressive language models -- manifested as repetitive token loops where the model becomes trapped in self-reinforcing attractors -- is a persistent pathology that existing decoding-time heuristics fail to address at its root cause. We present a principled framework that penalises or compensates anomalous confidence arising from collapsed generation patterns, by comparing a token's observed frequency against its corpus prior through an adjacent-conditional probability construction. The resulting self-normalising penalty ratio $R=f(m,n,p)/f(np,n,p)$ requires no ad hoc standardisation and admits a closed-form logit offset with zero approximation error. The correction is isolated from the loss gradient and accumulated into a frozen output-layer bias via exponential moving average, enabling deployment as a repair mechanism for models that have already collapsed without requiring intrusive modifications to standard training pipelines. Experimental validation on a 1.5B-parameter model demonstrates that the frozen-bias mechanism can rescue a model already trapped in a collapsed attractor, reducing 2-gram repetition from 0.073 to near 0 while preserving generation quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。