模型生成数据循环训练时，说得好却越来越离谱，需针对性防范。
Knowledge Collapse in LLMs: When Fluency Survives but Facts Fail under Recursive Synthetic Training
- 通过递归合成训练观察到三阶段知识退化，表面流畅但事实失真。
- 指令格式影响退化速度与模式，提示词不同结果差异显著。
- 针对特定领域生成数据可有效抵抗退化，兼顾效率与准确率。
大型语言模型因人工内容稀缺而越来越多依赖合成数据，但基于模型生成输出的递归训练会导致模型崩溃——一种威胁事实可靠性的退化过程。我们定义知识崩溃为一种独特的三阶段现象：事实准确性下降，而表面流畅性持续存在，产生“自信错误”的输出，在依赖准确性的领域带来重大风险。通过控制实验验证，发现崩溃轨迹和时机高度依赖指令格式，其条件性与提示词相关，区别于传统模型崩溃。我们提出领域特定合成训练作为针对性缓解策略，在保持计算效率的同时显著提升抗崩溃能力。评估框架结合模型中心指标与任务中心度量，可识别不同退化阶段，实现跨模型的可复现认知退化评估。研究揭示了崩溃动态的理论机制，并为知识密集型应用中可持续训练提供实践指导。
原文摘要 · Abstract (English)
Large language models increasingly rely on synthetic data due to human-written content scarcity, yet recursive training on model-generated outputs leads to model collapse, a degenerative process threatening factual reliability. We define knowledge collapse as a distinct three-stage phenomenon where factual accuracy deteriorates while surface fluency persists, creating "confidently wrong" outputs that pose critical risks in accuracy-dependent domains. Through controlled experiments with recursive synthetic training, we demonstrate that collapse trajectory and timing depend critically on instruction format, distinguishing instruction-following collapse from traditional model collapse through its conditional, prompt-dependent nature. We propose domain-specific synthetic training as a targeted mitigation strategy that achieves substantial improvements in collapse resistance while maintaining computational efficiency. Our evaluation framework combines model-centric indicators with task-centric metrics to detect distinct degradation phases, enabling reproducible assessment of epistemic deterioration across different language models. These findings provide both theoretical insights into collapse dynamics and practical guidance for sustainable AI training in knowledge-intensive applications where accuracy is paramount.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。