低困惑度不等于高质量,论文发现扩散语言模型的重复问题并提出简单修复方法。
Low Perplexity is Repetition: A One-Dimensional Self-Conditioning Attractor in Continuous Diffusion LMs
- 发现扩散语言模型在自回归反馈中存在一维收缩吸引子导致重复
- 提出的ACE方法移除单一方向噪声,使重复率接近人类水平
- 修复后生成质量更真实,且计算成本更低,适合实际部署
连续扩散语言模型(如ELF)报告了极低的生成困惑度(Gen-PPL)。我们发现:这类模型重复远超人类文本,而Gen-PPL反而奖励而非惩罚重复,因此其低分夸大了真实质量。去除重复后,ELF-B的Gen-PPL从19.5升至27.7;最小模型因重复最多反而获得最佳分数。问题根源在于自条件反馈环中沿单一方向的收缩吸引子。由于故障为一维,仅需一维修复,我们提出ACE(Attractor-Contrast-Escape):在每步反馈中减去该无标签方向。在105M模型上估计一次该方向,即可将重复率降至人类水平,同时保持竞争力,并可无缝迁移至342M和652M模型及不同采样器;相同方法亦适用于其他架构。由于Gen-PPL本身奖励重复,我们改用生成人写文本所需的算力衡量效率,结果显示ACE节省1.5–5倍算力。
原文摘要 · Abstract (English)
Continuous diffusion language models such as ELF report record-low generative perplexity (Gen-PPL). We find a catch: these models repeat far more than human text, and Gen-PPL rewards rather than penalizes that repetition, so its low scores overstate quality. Strip the repetition and ELF-B's Gen-PPL rises from $19.5$ to $27.7$; the smallest model even posts the best Gen-PPL because it repeats most. We trace the repetition to its source: a contractive attractor along a \emph{single direction} in the self-conditioning feedback loop, the loop that feeds each step's clean estimate into the next. Because the failure is one-dimensional, a one-dimensional fix suffices, and we propose one. \textbf{ACE} (Attractor-Contrast-Escape) subtracts that single, label-free direction from the feedback at each step. Estimated once on the $105$M model, the direction cuts repetition to near the human level while keeping quality competitive, and transfers near-unchanged to the $342$M and $652$M models and across samplers; the same recipe recovers useful directions on other architectures. Since Gen-PPL itself rewards repetition, we instead measure the compute each fix needs to produce human-clean text, where ACE is $1.5$--$5\times$ cheaper.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。