轻量级扩散编码模型CoDA实现高效代码生成，性能媲美更大模型。
CoDA: Coding LM via Diffusion Adaptation
- 用扩散模型做代码生成，结合中段微调与指令调整
- 17亿参数模型在代码评测上超过70亿参数的同类模型
- 开源训练流程与模型权重，助力轻量化研究
扩散语言模型具备双向上下文和填空生成能力，但实际系统仍显笨重。我们提出CoDA，一个基于TPU训练的17亿参数扩散编码器，采用全开源训练流程。CoDA结合大规模扩散预训练、以代码为中心的中段微调及指令微调，支持置信度引导采样，在推理延迟上保持竞争力。在Humaneval、MBPP和EvalPlus测试集上，CoDA-1.7B-Instruct的表现达到或超越参数量高达70亿的扩散模型。我们发布了模型检查点、评估工具和TPU训练管道，推动轻量化扩散式代码助手的研究进展。
原文摘要 · Abstract (English)
Diffusion language models promise bidirectional context and infilling capabilities that autoregressive coders lack, yet practical systems remain heavyweight. We introduce CoDA, a 1.7B-parameter diffusion coder trained on TPU with a fully open-source training pipeline. CoDA pairs large-scale diffusion pre-training with code-centric mid-training and instruction tuning, enabling confidence-guided sampling that keeps inference latency competitive. On Humaneval, MBPP, and EvalPlus, CoDA-1.7B-Instruct matches or surpasses diffusion models up to 7B parameters. Our release includes model checkpoints, evaluation harnesses, and TPU training pipelines to accelerate research on lightweight diffusion-based coding assistants.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。