无需训练，用参考文本实时引导扩散语言模型生成。
ILRR: Inference-Time Steering Method for Masked Diffusion Language Models
- 通过动态对齐生成序列与参考文本的内部激活，实现语义控制。
- 在相同计算开销下，属性控制准确率提升10%至60%。
- 支持长文本用短参考控制，适合需要灵活生成的场景。
离散扩散语言模型（DLMs）为文本生成提供了有前景的非自回归替代方案，但推理时的可控机制仍研究不足。现有方法包括采样级引导或轨迹优化。本文提出无需学习的迭代潜在表示精炼（ILRR）框架，仅需一个参考序列即可引导生成。ILRR 在去噪过程中动态对齐生成序列与参考序列的内部激活，捕捉并传递高层语义特征，可通过调节引导尺度灵活控制情感等属性。进一步提出空间调制引导，通过调节序列各段引导强度，实现以短参考控制长文本。实验证明，ILRR 在 LLaDA 与 MDLM 架构上均实现有效属性控制，每次去噪步骤仅增加一次并行前向传播，计算开销极小。在同等计算预算下，相比基线模型，属性准确率提升10%至60%，同时保持高质量生成。
原文摘要 · Abstract (English)
Discrete Diffusion Language Models (DLMs) offer a promising non-autoregressive alternative for text generation, yet effective mechanisms for inference-time control remain relatively underexplored. Existing approaches include sampling-level guidance procedures or trajectory optimization mechanisms. In this work, we introduce Iterative Latent Representation Refinement (ILRR), a learning-free framework for steering DLMs using a single reference sequence. ILRR guides generation by dynamically aligning the internal activations of the generated sequence with those of a given reference throughout the denoising process. This approach captures and transfers high-level semantic properties, with a tunable steering scale enabling flexible control over attributes such as sentiment. We further introduce Spatially Modulated Steering, an extension that enables steering long texts using shorter references by regulating guidance intensity across the sequence. Empirically, we demonstrate that ILRR achieves effective attribute steering on LLaDA and MDLM architectures with a minor computational overhead, requiring only one additional parallel forward pass per denoising step. Under the same compute budget, ILRR improves attribute accuracy over comparable baselines by 10$\%$ to 60$\%$ points, while maintaining high generation quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。