arXiv:2604.04215cs.CL2026-04被引 4

DARE统一了扩散语言模型的训练与评估流程,提升研究可复现性。

DARE: Diffusion Large Language Models Alignment and Reinforcement Executor

  • 构建统一框架整合多种微调与强化学习方法
  • 支持多个主流扩散模型家族,实现可复现基准测试
  • 适合希望快速比较和部署扩散模型后训练方法的研究者

扩散大语言模型(dLLMs)正成为主流自回归模型的有力替代,以迭代去噪和并行生成机制取代严格顺序的词元生成。然而其开源生态在模型家族及后训练流程间仍高度碎片化,尤其是强化学习目标、采样实现与评估脚本常以论文专属代码库形式发布。这导致研究迭代缓慢、复现工程负担重,且算法间难以公平比较。本文提出DARE(dLLMs对齐与强化执行器),一个面向dLLMs后训练与评估的开源框架。基于verl和OpenCompass构建,DARE在掩码与块扩散语言模型上统一了监督微调、参数高效微调、偏好优化及dLLM特有强化学习方法。覆盖LLaDA、Dream、SDAR及LLaDA2.x等代表性模型家族,提供广泛算法支持、可复现的基准评估与实际加速能力。大量实验证明,DARE可作为当前及未来dLLMs后训练方法开发、比较与部署的可复用研究基础。

原文摘要 · Abstract (English)

Diffusion large language models (dLLMs) are emerging as a compelling alternative to dominant autoregressive models, replacing strictly sequential token generation with iterative denoising and parallel generation dynamics. However, their open-source ecosystem remains fragmented across model families and, in particular, across post-training pipelines, where reinforcement learning objectives, rollout implementations and evaluation scripts are often released as paper-specific codebases. This fragmentation slows research iteration, raises the engineering burden of reproduction, and makes fair comparison across algorithms difficult. We present \textbf{DARE} (\textbf{d}LLMs \textbf{A}lignment and \textbf{R}einforcement \textbf{E}xecutor), an open framework for post-training and evaluating dLLMs. Built on top of verl~\cite{sheng2024hybridflow} and OpenCompass~\cite{2023opencompass}, DARE unifies supervised fine-tuning, parameter-efficient fine-tuning, preference optimization, and dLLM-specific reinforcement learning under a shared execution stack for both masked and block diffusion language models. Across representative model families including LLaDA, Dream, SDAR, and LLaDA2.x, DARE provides broad algorithmic coverage, reproducible benchmark evaluation, and practical acceleration. Extensive empirical results position that DARE serves as a reusable research substrate for developing, comparing, and deploying post-training methods for current and emerging dLLMs.

扩散模型语言模型强化学习框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。