arXiv:2603.30040cs.SEcs.AI2026-03

用轻量Transformer自动识别代码中可并行的循环,准确率超99%。

Automatic Identification of Parallelizable Loops Using Transformer-Based Source Code Representations

  • 基于DistilBERT的代码序列建模,无需手工特征
  • 跨10折交叉验证,平均准确率超99%,误报率低
  • 适合做编译器优化或自动化并行化工具的研发者

自动并行化仍是软件工程中的难题,尤其在识别现代多核架构上可安全并行执行的代码区域方面。传统静态分析技术如依赖分析和多面体模型对不规则或动态结构的代码处理效果不佳。本文提出一种基于Transformer的方法,通过区分独立(可并行)循环与非独立循环来评估代码的并行化潜力。采用DistilBERT处理源码序列,结合子词分词,捕捉上下文的语法与语义模式,无需人工设计特征。在合成生成循环与真实代码手动标注的数据集上进行评估,使用10折交叉验证和多种性能指标。结果表明,该方法表现稳定,平均准确率超过99%,且误报率极低,展现出强鲁棒性与可靠性。相比以往基于词元的方法,本方案简化了预处理流程,提升了泛化能力,同时保持计算效率。这些发现表明,轻量级Transformer模型在循环级别识别并行化机会方面具有实际应用潜力。

原文摘要 · Abstract (English)

Automatic parallelization remains a challenging problem in software engineering, particularly in identifying code regions where loops can be safely executed in parallel on modern multi-core architectures. Traditional static analysis techniques, such as dependence analysis and polyhedral models, often struggle with irregular or dynamically structured code. In this work, we propose a Transformer-based approach to classify the parallelization potential of source code, focusing on distinguishing independent (parallelizable) loops from undefined ones. We adopt DistilBERT to process source code sequences using subword tokenization, enabling the model to capture contextual syntactic and semantic patterns without handcrafted features. The approach is evaluated on a balanced dataset combining synthetically generated loops and manually annotated real-world code, using 10-fold cross-validation and multiple performance metrics. Results show consistently high performance, with mean accuracy above 99\% and low false positive rates, demonstrating robustness and reliability. Compared to prior token-based methods, the proposed approach simplifies preprocessing while improving generalization and maintaining computational efficiency. These findings highlight the potential of lightweight Transformer models for practical identification of parallelization opportunities at the loop level.

代码分析Transformer并行化编译器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。