解决大模型生成重复内容的生产难题,提出三种实用方案。
Solving LLM Repetition Problem in Production: A Comprehensive Study of Multiple Solutions
- 用束搜索+提前终止消除重复,通用且有效。
- 引入存在惩罚项,专治规则生成重复问题。
- 通过微调优化模型,从根本上解决三类重复。
大语言模型在实际批量代码解释任务中常出现重复生成问题,导致系统性能下降甚至崩溃。本文识别出三类重复模式:业务规则生成重复、方法调用关系分析重复、PlantUML语法生成重复。基于马尔可夫模型的理论分析表明,根本原因在于贪婪解码无法跳出重复循环,且存在自增强效应。实验验证了三种可行方案:(1) 束搜索配合 early_stopping=True 可通用解决所有重复模式;(2) presence_penalty 超参数对第一类重复问题有效;(3) 直接偏好优化(DPO)微调提供全场景模型级解决方案。本工作结合一线生产经验与充分实验验证,系统分析了重复机制,明确了 early_stopping 是束搜索有效性的关键参数,并提供了经真实环境验证的可落地解决方案。
原文摘要 · Abstract (English)
The repetition problem, where Large Language Models (LLMs) continuously generate repetitive content without proper termination, poses a critical challenge in production deployments, causing severe performance degradation and system stalling. This paper presents a comprehensive investigation and multiple practical solutions for the repetition problem encountered in real-world batch code interpretation tasks. We identify three distinct repetition patterns: (1) business rule generation repetition, (2) method call relationship analysis repetition, and (3) PlantUML diagram syntax generation repetition. Through rigorous theoretical analysis based on Markov models, we establish that the root cause lies in greedy decoding's inability to escape repetitive loops, exacerbated by self-reinforcement effects. Our comprehensive experimental evaluation demonstrates three viable solutions: (1) Beam Search decoding with early_stopping=True serves as a universal post-hoc mechanism that effectively resolves all three repetition patterns; (2) presence_penalty hyperparameter provides an effective solution specifically for BadCase 1; and (3) Direct Preference Optimization (DPO) fine-tuning offers a universal model-level solution for all three BadCases. The primary value of this work lies in combining first-hand production experience with extensive experimental validation. Our main contributions include systematic theoretical analysis of repetition mechanisms, comprehensive evaluation of multiple solutions with task-specific applicability mapping, identification of early_stopping as the critical parameter for Beam Search effectiveness, and practical production-ready solutions validated in real deployment environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。