用Transformer模型实现编程语言与高性能计算扩展间的精准无监督翻译。
CodeRosetta: Pushing the Boundaries of Unsupervised Code Translation for Parallel Programming
- 设计专用编码器-解码器架构，结合定制预训练目标捕捉并行代码结构。
- 在C++转CUDA任务上提升2.9 BLEU和1.72 CodeBLEU，编译成功率高6.05%。
- 首次实现Fortran到并行C++的无监督翻译，优于现有闭源与开源模型。
大型语言模型（LLMs）的进展重新激发了自动编程语言翻译的兴趣。尽管编码器-解码器Transformer模型在跨语言翻译中表现良好，但针对高性能计算（HPC）扩展的翻译仍面临挑战，如复杂的并行语义。本文提出CodeRosetta，一种专为编程语言及其HPC扩展间翻译设计的编码器-解码器Transformer模型。该模型在C++到CUDA和Fortran到C++的翻译任务上进行评估，采用定制化的预训练与训练目标，有效捕捉代码语义与并行结构细节，实现双向翻译。结果表明，CodeRosetta在C++到CUDA翻译中较先进基线提升2.9 BLEU与1.72 CodeBLEU，编译准确率提高6.05%；相比通用闭源LLMs，分别提升22.08 BLEU与14.39 CodeBLEU，编译准确率高2.75%。此外，其在Fortran到并行C++翻译中表现优异，据我们所知是首个实现该任务的编码器-解码器模型，相较闭源与开源模型至少提升4.63 CodeBLEU。
原文摘要 · Abstract (English)
Recent advancements in Large Language Models (LLMs) have renewed interest in automatic programming language translation. Encoder-decoder transformer models, in particular, have shown promise in translating between different programming languages. However, translating between a language and its high-performance computing (HPC) extensions remains underexplored due to challenges such as complex parallel semantics. In this paper, we introduce CodeRosetta, an encoder-decoder transformer model designed specifically for translating between programming languages and their HPC extensions. CodeRosetta is evaluated on C++ to CUDA and Fortran to C++ translation tasks. It uses a customized learning framework with tailored pretraining and training objectives to effectively capture both code semantics and parallel structural nuances, enabling bidirectional translation. Our results show that CodeRosetta outperforms state-of-the-art baselines in C++ to CUDA translation by 2.9 BLEU and 1.72 CodeBLEU points while improving compilation accuracy by 6.05%. Compared to general closed-source LLMs, our method improves C++ to CUDA translation by 22.08 BLEU and 14.39 CodeBLEU, with 2.75% higher compilation accuracy. Finally, CodeRosetta exhibits proficiency in Fortran to parallel C++ translation, marking it, to our knowledge, as the first encoder-decoder model for this complex task, improving CodeBLEU by at least 4.63 points compared to closed-source and open-code LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。