arXiv:2508.10995cs.CLcs.LG2025-08中稿 · as a main conferen…被引 1

用验证器提升文本风格迁移质量,无需复杂训练。

Improving Text Style Transfer using Masked Diffusion Language Models with Inference-time Scaling

  • 在去噪过程中引入外部验证器,动态优化生成候选。
  • 结合现成嵌入模型,显著提升生成质量。
  • 适合追求高质量非自回归生成的自然语言研究者。

掩码扩散语言模型(MDMs)因其可扩展性和易训练性,已成为离散数据生成的前沿非自回归框架。扩散模型通过增加去噪步骤或使用外部验证器指导生成,显著提升生成质量。本文提出一种基于验证器的推理阶段缩放方法,帮助MDM在去噪过程中找到更优的生成候选。实验表明,该方法适用于标准文本风格迁移任务,并优于传统自回归语言模型。此外,仅使用现成预训练嵌入模型构建的简单软值验证器,在现有无分类器引导设置上即带来显著质量提升。

原文摘要 · Abstract (English)

Masked diffusion language models (MDMs) have recently gained traction as a viable generative framework for natural language. This can be attributed to its scalability and ease of training compared to other diffusion model paradigms for discrete data, establishing itself as the state-of-the-art non-autoregressive generator for discrete data. Diffusion models, in general, have shown excellent ability to improve the generation quality by leveraging inference-time scaling either by increasing the number of denoising steps or by using external verifiers on top of the outputs of each step to guide the generation. In this work, we propose a verifier-based inference-time scaling method that aids in finding a better candidate generation during the denoising process of the MDM. Our experiments demonstrate the application of MDMs for standard text-style transfer tasks and establish MDMs as a better alternative to autoregressive language models. Additionally, we show that a simple soft-value-based verifier setup for MDMs using off-the-shelf pre-trained embedding models leads to significant gains in generation quality even when used on top of typical classifier-free guidance setups in the existing literature.

文本生成扩散模型风格迁移

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。