提出基于真实答案的排序机制,让文本生成更懂先难后易。
Where-to-Unmask: Ground-Truth-Guided Unmasking Order Learning for Masked Diffusion Language Models
- 用真实词与最强干扰词的概率差定义位置优先级
- 在逻辑推理任务上提升生成准确率,最高达18.7%改善
- 可插件式接入现有模型,无需重训练生成部分
掩码扩散语言模型通过逐步填充掩码标记生成文本,每一步需决定哪位置解封(where-to-unmask)和填什么词(what-to-unmask)。现有方法通常仅优化词预测,而解封顺序依赖启发式或强化学习,代价高昂。本文提出Gt-Margin——一种基于真实词的概率边际得分,衡量每个位置的解封难易程度。该得分可生成最优解封顺序:优先解封较易位置。实验表明,使用此最优顺序显著提升生成质量,尤其在逻辑推理任务中表现突出。进一步,我们设计一个监督式解封规划器,通过学习排序模仿该最优顺序,在不修改原生成模型的前提下,实现更优解封策略,有效提升推理准确性。
原文摘要 · Abstract (English)
Masked Diffusion Language Models (MDLMs) generate text by iteratively filling masked tokens, requiring two coupled decisions at each step: which positions to unmask (where-to-unmask) and which tokens to place (what-to-unmask). While standard MDLM training directly optimizes token prediction (what-to-unmask), inference-time unmasking orders (where-to-unmask) are typically determined by heuristic confidence measures or trained through reinforcement learning with costly on-policy rollouts. To address this, we introduce Gt-Margin, a position-wise score derived from ground-truth tokens, defined as the probability margin between the correct token and its strongest alternative. Gt-Margin yields an oracle unmasking order that prioritizes easier positions first under each partially masked state. We demonstrate that leveraging this oracle unmasking order significantly enhances final generation quality, particularly on logical reasoning benchmarks. Building on this insight, we train a supervised unmasking planner via learning-to-rank to imitate the oracle ordering from masked contexts. The resulting planner integrates into standard MDLM sampling to select where-to-unmask, improving reasoning accuracy without modifying the token prediction model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。