TESS 2是性能超越主流扩散模型的通用指令跟随语言模型。
TESS 2: A Large-Scale Generalist Diffusion Language Model
- 先用自回归模型继续预训练，再指令微调，提升扩散语言模型能力。
- 在多个任务上表现优于现有扩散模型，接近甚至超过强自回归模型。
- 提出推理时奖励引导机制，无需重训练即可控制输出质量。
我们提出TESS 2，一种通用指令遵循的扩散语言模型，其性能超越当前主流指令微调的扩散模型，并可媲美甚至超过强大的自回归（AR）模型。训练过程首先通过交叉熵作为扩散损失，对强自回归模型进行持续预训练，随后进行进一步的指令微调。我们发现适应性训练及基座模型选择对训练高质量的指令跟随扩散模型至关重要。此外，我们提出一种新颖且模块化的推理时奖励引导方法，可在不需重新训练模型的前提下对输出进行对齐。最后，我们表明TESS 2在推理时增加计算量可进一步提升性能，凸显了扩散语言模型在推理时可精细调控计算资源的优势。代码与模型已开源：https://github.com/hamishivi/tess-2。
原文摘要 · Abstract (English)
We introduce TESS 2, a general instruction-following diffusion language model that outperforms contemporary instruction-tuned diffusion models, as well as matches and sometimes exceeds strong autoregressive (AR) models. We train TESS 2 by first adapting a strong AR model via continued pretraining with the usual cross-entropy as diffusion loss, and then performing further instruction tuning. We find that adaptation training as well as the choice of the base model is crucial for training good instruction-following diffusion models. We further propose reward guidance, a novel and modular inference-time guidance procedure to align model outputs without needing to train the underlying model. Finally, we show that TESS 2 further improves with increased inference-time compute, highlighting the utility of diffusion LMs in having fine-grained controllability over the amount of compute used at inference time. Code and models are available at https://github.com/hamishivi/tess-2.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。