arXiv:2509.20380cs.SEcs.AI2025-09

用大模型自动加OpenACC指令,准确率达87%。

ACCeLLiuM: Supervised Fine-Tuning for Automated OpenACC Pragma Generation

  • 用4033个真实代码片段微调两个开源大模型,专攻数据并行循环的OpenACC指令生成。
  • 在测试集上,87%的指令类型正确,50%完全匹配标准答案。
  • 生成的指令虽不完全一致,但常包含更优控制选项,实用性强。

GPU普及带来硬件与编程框架复杂性的上升。基于指令的并行编程标准如OpenACC虽简化了编程,但仍需专业知识才能有效使用。本文提出ACCeLLiuM,两个针对数据并行循环生成专家级OpenACC指令的开源大语言模型,以及用于训练的监督微调数据集。该数据集从公开GitHub C/C++仓库中挖掘出4,033个OpenACC pragma-循环对,其中3,223用于训练,810用于测试。实验表明,基础大模型在生成有效OpenACC指令方面表现不佳,而经过ACCeLLiuM数据集微调的模型,在保留测试集上对87%的数据并行循环生成了正确的指令类型,对50%的案例生成了完全匹配的指令(包括指令、子句、顺序和变量)。即使不完全一致,生成的指令也常包含正确子句,或以不同顺序排列,或增加额外子句,从而实现对并行执行、数据移动和并发的更精细控制,具有实际应用价值。通过公开发布代码、模型和数据集,我们希望为大模型驱动的OpenACC指令生成建立可复现基准,降低串行程序自动迁移到GPU的门槛。

原文摘要 · Abstract (English)

The increasing ubiquity of GPUs is accompanied by the increasing complexity of their hardware and parallel programming frameworks. Directive-based parallel programming standards like OpenACC simplify GPU programming to some extent by abstracting away low-level complexities, but a fair amount of expertise is still required in order to use those directives effectively. We introduce ACCeLLiuM, two open weights Large Language Models specifically fine-tuned for generating expert OpenACC directives for data-parallel loops, along with the supervised fine-tuning dataset that was used to train them. The ACCeLLiuM SFT dataset contains 4,033 OpenACC pragma-loop pairs mined from public GitHub C/C++ repositories, with 3,223 pairs for training and 810 for testing. Experimental evaluations show a pronounced performance gap in generating correct OpenACC pragmas between base LLMs and our fine-tuned versions. On the held-out test set, base LLMs fail to consistently generate valid pragmas, whereas LLMs fine-tuned on the ACCeLLiuM dataset generate valid pragmas with the correct directive type for $87\%$ of the data-parallel loops, and exact pragmas--including directives, clauses, clause order, and clause variables--for $50\%$ of the cases. Even when not exact, generated pragmas frequently incorporate the correct clauses in a different order than the ground-truth label, or include additional clauses that enable finer control over parallel execution, data movement, and concurrency, offering practical value beyond strict string-matching. By publicly releasing the code, models, and dataset as ACCeLLiuM we hope to establish a reproducible benchmark for LLM-powered OpenACC pragma generation, and lower the barrier to automated GPU offloading of serially written programs.

代码生成GPU优化大模型OpenACC

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。