揭示扩散语言模型中顺序依赖的根源，提出可诊断的非保守场分析框架。
Path-Dependent Denoising: A Non-Conservative Field Perspective on Order Collapse in Diffusion Language Models
- 用伪联合分布与局部循环定义非保守场，刻画顺序依赖性
- 发现顺序漂移源于条件不兼容，而非估计误差或依赖偏差
- 提供仅需推理的诊断工具，判断解码是否真正无序
扩散语言模型（DLMs）为自回归生成提供结构替代：去噪可在任意顺序或并行更新词元，而非固定左到右链式。实践中，快速解码仍高度依赖顺序，常趋向自回归轨迹。我们追溯此矛盾源于条件兼容性问题：在每个逆时序步骤中，DLM 提供未定词元上的局部去噪条件。当这些局部条件能组合成顺序无关的伪联合分布时，任意顺序去噪才成立。本文通过定义顺序诱导的伪联合分布与局部去噪循环（交换一对未定位置所得伪联合对数比），形式化该视角。循环为零时条件兼容；全局顺序差异可分解为相邻交换路径上局部循环之和。进一步分离了由不兼容导致的路径依赖、并行更新中的条件依赖误差及顺序特异性估计误差。所提框架可实现仅推理阶段的诊断，检验DLM解码是否真正无序。
原文摘要 · Abstract (English)
Diffusion language models (DLMs) offer a structural alternative to autoregressive generation: denoising can update tokens in arbitrary orders or in parallel rather than along a fixed left-to-right chain. In practice, fast DLM decoding remains strongly order-sensitive and often drifts toward autoregressive-like trajectories. We trace this tension to compatibility. At each reverse-time step, a DLM provides local denoising conditionals over the unresolved tokens. Arbitrary-order denoising becomes well defined when these local conditionals compose into order-invariant pseudo-joints. We formalize this view by defining order-induced pseudo-joints and a local denoising circulation: the log-ratio between the two pseudo-joints obtained by swapping a pair of unresolved positions. This circulation is zero under compatible conditionals, and global order gaps decompose into sums of local circulations along adjacent swaps. We further separate incompatibility-driven path dependence from conditional-dependence error in parallel updates and from order-specific estimation error. The resulting framework provides inference-only diagnostics for testing when DLM decoding is genuinely order-free.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。