发现顺序无关语言模型的生成结果受推理顺序影响,提出用置信度方差诊断解码路径质量。
Decoding in Order-Agnostic Language Models: Chain-Rule Deviation and Uniform Spreading
- 通过置信度轨迹形状诊断解码路径,发现均匀分布时恢复效果最佳
- 改变提示顺序可使对数似然变化达0.49纳特/词元,说明结果依赖路径
- 建议联合报告平均置信度与置信度方差,便于比较不同解码策略
顺序无关语言模型(OALMs),包括离散扩散语言模型(dLLMs),在任意条件集下训练预测被掩码的词元,可在推理时以任意顺序生成或评分序列。在LLaDA-2.1中,我们报告三个发现:第一,学习到的条件分布并非精确的联合分布因子分解:仅改变揭示顺序即可导致目标对数似然变化达0.49纳特/词元,因此似然值同时混杂了内容难度与路径相关伪影;第二,尽管置信度优先(CF)解码是顺序无关的,其揭示顺序在内容词元上仍接近从左到右(L2R);第三,我们提出一种基于置信度轨迹形态的互补诊断方法。一个均匀扩散定理表明,在固定总似然条件下,每步置信度均匀分布时目标恢复能力最强;由此产生的偏离促使使用$\mathrm{Var}(\log q_t)$作为比较解码路径的诊断指标。在C4及四个下游基准上,低方差能有效区分结构化路径与随机顺序,且方差与下游正确性始终相关。这些结果支持在比较OALM解码路径时联合报告均值置信度与置信度方差。
原文摘要 · Abstract (English)
Order-agnostic language models (OALMs), including discrete diffusion language models (dLLMs), are trained to predict masked tokens under arbitrary conditioning sets, allowing sequences to be generated or scored under arbitrary reveal orders at inference time. In LLaDA-2.1, we report three findings. First, the learned conditionals are not exact factorizations of a coherent joint distribution: changing only the reveal order shifts target log-likelihood by up to 0.49 nats/token, so likelihood alone mixes content difficulty with path-dependent artifacts. Second, although confidence-first (CF) decoding is order-agnostic, its reveal orders are close to left-to-right (L2R) on content tokens. Third, we propose a complementary diagnostic based on the shape of the confidence trace. A uniform-spreading theorem shows that, at fixed total likelihood, target recoverability is maximized when per-step confidence is spread uniformly; the resulting deviation motivates $\mathrm{Var}(\log q_t)$ as a diagnostic for comparing decoding paths. Across C4 and four downstream benchmarks, low variance separates structured paths from random ordering, and variance is consistently associated with downstream correctness. These results support reporting mean confidence and confidence variance jointly when comparing OALM decoding paths.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。