用大模型自动生成代码片段对齐数据,提升代码翻译准确率
Automated Snippet-Alignment Data Augmentation for Code Translation
- 用LLM自动构建细粒度代码片段对齐数据
- 两阶段训练使代码翻译准确率最高提升3.78%(pass@k)
- 适合需要高质量代码翻译数据的开发与研究者
代码翻译旨在将源语言代码转换为目标语言,广泛应用于软件开发。近年来大型语言模型在代码翻译中表现突出,而并行语料库在训练中起关键作用。并行数据可分为程序级对齐(PA)和片段级对齐(SA)数据。尽管PA数据具备完整上下文,利于语义对齐学习,但其较长长度可能限制细粒度训练信号;而SA数据更短,有利于精细对齐。由于并行数据有限,研究者探索多种增强方法。以往工作多聚焦于PA数据增强。本文提出一种基于LLM自动生成SA数据的自动化增强方法,并设计简单有效的两阶段训练策略,相比仅微调PA数据,显著提升模型性能。在TransCoder-test上的实验表明,结合增强后的SA数据与两阶段训练,相较基线实现一致提升,最高达3.78%(pass@k)。
原文摘要 · Abstract (English)
Code translation aims to translate the code from its source language to the target language and is used in various software development scenarios. Recent developments in Large Language Models (LLMs) have showcased their capabilities in code translation, and parallel corpora play a crucial role in training models for code translation. Parallel corpora can be categorized into program-alignment (PA) and snippet-alignment (SA) data. Although PA data has complete context and is suitable for semantic alignment learning, it may not provide adequate fine-grained training signals due to its extended length, while the brevity of SA data enables more fine-grained alignment learning. Due to limited parallel corpora, researchers explore several augmentation methods for code translation. Previous studies mainly focus on augmenting PA data. In this paper, we propose a data augmentation method that leverages LLMs to generate SA data automatically. To fully leverage both PA data and SA data, we explore a simple yet effective two-stage training strategy, which consistently enhances model performance compared to fine-tuning solely on PA data. Experiments on TransCoder-test demonstrate that our augmented SA data combined with the two-stage training approach yields consistent improvements over the baseline, achieving a maximum gain of 3.78% on pass@k.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。