arXiv:2503.09790cs.CLcs.LG2025-03NeurIPS被引 13

让生成文本自动满足约束条件,无需重训练

Constrained Discrete Diffusion

  • 在扩散采样中嵌入可微约束优化,直接控制生成结果
  • 多任务测试中零违规,同时保持语言流畅与创新性
  • 适合需要安全、合规或逻辑一致性的生成场景

离散扩散模型通过逐步去噪从类别噪声分布中构建序列。除了生成连贯自然语言的能力快速提升外,这类模型还为强制执行序列级约束提供了新机会,这是当前自回归模型无法原生支持的。本文提出约束离散扩散(CDD),将可微约束优化融入扩散过程,确保生成序列符合约束、逻辑规则或安全要求。不同于依赖事后过滤或模型重训练的生成方法,CDD 直接在离散扩散采样过程中施加约束,实现无需训练的有效控制。在毒性控制文本生成、属性约束分子设计和指令约束文本补全等任务中的实验表明,CDD 在多种场景下实现零约束违规,同时保持语言流畅性、新颖性和连贯性,优于自回归及现有离散扩散方法。

原文摘要 · Abstract (English)

Discrete diffusion models are a class of generative models that construct sequences by progressively denoising samples from a categorical noise distribution. Beyond their rapidly growing ability to generate coherent natural language, these models present a new and important opportunity to enforce sequence-level constraints, a capability that current autoregressive models cannot natively provide. This paper capitalizes on this opportunity by introducing Constrained Discrete Diffusion (CDD), a novel integration of differentiable constraint optimization within the diffusion process to ensure adherence to constraints, logic rules, or safety requirements for generated sequences. Unlike conventional text generators that often rely on post-hoc filtering or model retraining for controllable generation, CDD directly imposes constraints into the discrete diffusion sampling process, resulting in a training-free and effective approach. Experiments in toxicity-controlled text generation, property-constrained molecule design, and instruction-constrained text completion demonstrate that CDD achieves zero constraint violations in a diverse array of tasks while preserving fluency, novelty, and coherence while outperforming autoregressive and existing discrete diffusion approaches.

离散扩散可控生成约束满足

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。