dLLM统一扩散语言建模组件，让研究者轻松复现与构建新模型。
dLLM: Simple Diffusion Language Modeling
- 提供标准化训练、推理与评估流程，整合主流DLM核心组件。
- 支持从零构建小规模DLM，可在普通设备上运行，提供预训练权重。
- 开源框架+可复现配方，加速扩散语言模型的实验与创新。
尽管扩散语言模型（DLMs）发展迅速，但许多近期模型都采用一组共享组件。这些组件分散在非标准的研究代码库中或缺乏透明实现，导致复现和扩展困难。随着领域加速发展，亟需一个统一框架来标准化核心组件，同时保持对新方法和架构的灵活性。为此，我们提出dLLM，一个开源框架，统一了扩散语言建模的核心环节——训练、推理与评估，并支持对新设计的便捷定制。通过dLLM，用户可基于标准化流程复现、微调、部署和评估如LLaDA和Dream等开源大DLM。框架还提供最小化、可复现的配方，使研究人员能使用有限算力从头构建小型DLM，包括将任意BERT类编码器或自回归语言模型转换为DLM。我们还公开了这些小型DLM的检查点，以提升模型可及性并推动未来研究。
原文摘要 · Abstract (English)
Although diffusion language models (DLMs) are evolving quickly, many recent models converge on a set of shared components. These components, however, are distributed across ad-hoc research codebases or lack transparent implementations, making them difficult to reproduce or extend. As the field accelerates, there is a clear need for a unified framework that standardizes these common components while remaining flexible enough to support new methods and architectures. To address this gap, we introduce dLLM, an open-source framework that unifies the core components of diffusion language modeling -- training, inference, and evaluation -- and makes them easy to customize for new designs. With dLLM, users can reproduce, finetune, deploy, and evaluate open-source large DLMs such as LLaDA and Dream through a standardized pipeline. The framework also provides minimal, reproducible recipes for building small DLMs from scratch with accessible compute, including converting any BERT-style encoder or autoregressive LM into a DLM. We also release the checkpoints of these small DLMs to make DLMs more accessible and accelerate future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。