针对扩散语言模型推理能力弱的问题，提出自适应微调方法提升性能。
Learnability-Informed Fine-Tuning of Diffusion Language Models
- 根据输入掩码程度动态选择难易词汇进行微调
- 在六个推理基准上表现优于基线，最高提升3倍
- 适合需要增强逻辑推理的扩散模型应用
我们旨在提升扩散语言模型（DLMs）的推理能力。尽管监督微调（SFT）是自回归模型常用的后训练方法，但在DLMs中使用时面临挑战，甚至可能损害性能，但其根本原因尚未深入研究。我们的分析表明，传统SFT忽略了学习能力，即何时以及哪些词可被有效学习：当输入大部分被掩码时，罕见词难以学习；而当输入大多未被掩码时，常见词易于学习，价值较低。受此启发，我们提出LIFT——一种基于SFT的高效后训练算法。LIFT在多数输入被掩码时学习困难词，在更多上下文可用时学习简单词，从而与不同扩散时间步的信息状态对齐。实验结果表明，LIFT在六个推理基准上均优于现有SFT基线，在AIME'24和AIME'25上实现最高达3倍的相对提升。代码已公开于https://github.com/divelab/LIFT。
原文摘要 · Abstract (English)
We aim to improve the reasoning capabilities of diffusion language models (DLMs). While SFT is a popular post-training recipe for autoregressive models, its use in DLMs faces challenges and can even hurt performance, though the underlying causes remain understudied. Our analysis reveals that vanilla SFT overlooks learnability, namely what and when tokens are learned. Specifically, rare tokens are difficult to learn when most of the input is masked, whereas it is straightforward and thus of little value to learn common tokens when most of the input is unmasked. Motivated by our analysis, we propose LIFT, an efficient SFT-based post-training algorithm for DLMs. LIFT learns easy tokens when most of the input is masked and hard tokens when more context is available, thus aligning the training with the information available at different diffusion time steps. Our results show that LIFT outperforms existing SFT baselines across six reasoning benchmarks, achieving up to a 3x relative gain on AIME'24 and AIME'25. Our code is publicly available at https://github.com/divelab/LIFT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。