根据模型置信度动态调整无条件输入，提升生成准确性。
Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking
- 通过识别低置信度词元并临时遮蔽，生成动态无条件输入。
- 在GPQA上比标准CFG提升3.9分，显著改善生成质量。
- 适合需要精准控制的文本生成任务，如复杂问答和长文创作。
Classifier-Free Guidance (CFG) 通过插值条件与无条件预测显著提升生成模型的可控性。然而，标准CFG通常使用静态无条件输入，在迭代生成过程中模型不确定性动态变化时表现不佳。本文提出自适应无条件引导（A-CFG），利用模型即时预测置信度动态调整无条件输入。在每一步迭代的掩码扩散语言模型中，A-CFG识别当前生成序列中模型置信度较低的词元，并暂时重新遮蔽它们，从而构建局部动态的无条件输入。该机制将引导作用精准聚焦于模糊区域，提升引导效率。我们将A-CFG集成至当前最先进的掩码扩散语言模型，在多个语言生成基准上验证其有效性。实验表明，A-CFG相比标准CFG有显著提升，例如在GPQA上取得3.9分的增益。本工作揭示了在迭代生成中根据模型不确定性动态调整引导机制的优势。
原文摘要 · Abstract (English)
Classifier-Free Guidance (CFG) significantly enhances controllability in generative models by interpolating conditional and unconditional predictions. However, standard CFG often employs a static unconditional input, which can be suboptimal for iterative generation processes where model uncertainty varies dynamically. We introduce Adaptive Classifier-Free Guidance (A-CFG), a novel method that tailors the unconditional input by leveraging the model's instantaneous predictive confidence. At each step of an iterative (masked) diffusion language model, A-CFG identifies tokens in the currently generated sequence for which the model exhibits low confidence. These tokens are temporarily re-masked to create a dynamic, localized unconditional input. This focuses CFG's corrective influence precisely on areas of ambiguity, leading to more effective guidance. We integrate A-CFG into a state-of-the-art masked diffusion language model and demonstrate its efficacy. Experiments on diverse language generation benchmarks show that A-CFG yields substantial improvements over standard CFG, achieving, for instance, a 3.9 point gain on GPQA. Our work highlights the benefit of dynamically adapting guidance mechanisms to model uncertainty in iterative generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。