让语言模型自我生成错误来训练纠错能力，提升编辑精度。
Self-Generated Error Training for Token Editing in Diffusion Language Models
- 用模型自身生成的错误作为训练数据，避免训练与推理不一致。
- 在多个基准上提升编辑准确率，减少错误修正强度。
- 适合需要精准文本修改的应用，如对话系统、内容生成。
令牌到令牌（T2T）编辑允许在块扩散解码过程中修改已确定的令牌。现有方法在随机词汇错误上训练编辑器，但在推理时，编辑器面对的是模型自身流畅且高置信度的草稿错误。我们研究了这一训练-推理差异，提出自生成T2T：先无梯度草稿生成，用预测令牌填充掩码位置，再在第二次通过中以这些自生成错误为监督进行恢复。我们在LLaDA2.1-mini上通过短时LoRA持续预训练实现该更新，并在官方Q-Mode T2T流程下评估，使用不变的推理参数。该方法普遍提高准确性，同时降低编辑强度，缓解最终数字转录错误等失败模式，以及在简短事实回答前的过度自我修正问题。
原文摘要 · Abstract (English)
Token-to-token (T2T) editing lets LLaDA2.1 revise committed tokens during block-diffusion decoding. The released recipe trains this editor on random vocabulary corruptions, but at inference the editor sees the model's own fluent, high-confidence draft errors instead. We study this training-inference mismatch and propose self-generated T2T, which performs a no-gradient draft pass, fills masked positions with predicted tokens, and supervises recovery in a second pass under these self-generated corruptions. We implement the update as a short LoRA continued-pretraining pass on LLaDA2.1-mini and evaluate on several benchmarks under the official Q-Mode T2T procedure with unchanged inference parameters. The method generally improves accuracy while reducing T2T edit intensity, mitigating failure modes such as final-digit transcription errors after otherwise correct reasoning and excessive self-correction before short factual answers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。