T-star通过强化学习渐进扩展掩码块大小,提升解码并行性。
T$^\star$: Progressive Block Scaling for Masked Diffusion Language Models Through Trajectory Aware Reinforcement Learning
- 基于TraceRL设计训练流程,逐步增大掩码块尺寸
- 在数学推理任务上保持低性能损失,支持高并行解码
- 可能收敛到新解码策略,效果相当但路径不同
我们提出T$^\star$,一种基于TraceRL的训练课程,用于在掩码扩散语言模型(MDMs)中实现掩码块大小的渐进式扩展。从自回归初始化的小块MDM出发,T$^\star$平稳过渡到更大的块,使解码具有更高并行性,同时在数学推理基准上仅造成轻微性能下降。进一步分析表明,T$^\star$可能实际收敛至一种替代解码策略,其性能相当。
原文摘要 · Abstract (English)
We present T$^\star$, a simple TraceRL-based training curriculum for progressive block-size scaling in masked diffusion language models (MDMs). Starting from an AR-initialized small-block MDM, T$^\star$ transitions smoothly to larger blocks, enabling higher-parallelism decoding with minimal performance degradation on math reasoning benchmarks. Moreover, further analysis suggests that T$^\star$ may actually converge to an alternative decoding schedule that achieves comparable performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。