统一序列与树结构生成，提升代码生成质量
UniGenCoder: Merging Seq2Seq and Seq2Tree Paradigms for Unified Code Generation
- 共享编码器+动态选择器，融合序列与树两种生成方式
- 在文本转代码和代码转代码任务上均超越单一范式
- 适合需要高精度代码生成的研究者与开发者
基于深度学习的代码生成已彻底改变开发者编程方式。现有方法主要采用序列到序列（Seq2Seq）或序列到树（Seq2Tree）范式：前者将目标代码视为词元序列生成，后者则输出一系列操作动作。尽管两者直观互补，但尚未被有效结合。通过对比两种范式生成结果，我们发现融合具有显著潜力。本文提出UniGenCoder，用于统一代码相关生成任务，包含共享编码器、共享解码器（仅新增少量参数）及动态选择最优范式的选择器。训练阶段先采用多任务学习与知识蒸馏促进两范式间知识迁移，再通过对比学习训练选择器。在文本转代码与代码转代码任务上的实验表明，该模型显著有效。代码已开源：https://github.com/DeepLearnXMU/UniGenCoder。
原文摘要 · Abstract (English)
Deep learning-based code generation has completely transformed the way developers write programs today. Existing approaches to code generation have focused either on the Sequence-to-Sequence paradigm, which generates target code as a sequence of tokens, or the Sequence-to-Tree paradigm, which outputs code as a sequence of actions. While these two paradigms are intuitively complementary, their combination has not been previously explored. By comparing the code generated under these two paradigms, we find that integrating them holds significant potential. In this paper, we propose UniGenCoder for code-related generation tasks, which consists of a shared encoder, a shared decoder with a minimal set of additional parameters to unify two paradigms, and a selector that dynamically chooses optimal paradigm for each instance. Also, during the model training, we first perform the multi-task learning and distillation strategies to facilitate knowledge transfer between two paradigms, and then leverage contrastive learning to train the selector. Experimental results on the text-to-code and code-to-code generation tasks demonstrate the effectiveness of our proposed model. We release our code at https://github.com/DeepLearnXMU/UniGenCoder.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。