用动态思维优化静态代码表示,提升编译器优化预测能力
Behavioral Embeddings of Programs: A Quasi-Dynamic Approach for Optimization Prediction
- 通过优化序列探测程序行为变化,生成可量化的动态特征谱
- 在两个编译优化任务上超越现有静态基线模型,准确率显著提升
- 适合编译器优化研究者与高性能计算开发者参考
学习程序的有效数值表示(嵌入)是将机器学习应用于自动化和增强编译器优化的基础。当前范式面临两难:静态表示(基于源码或中间表示)高效且确定,但难以揭示程序在复杂代码变换下的行为演变;动态表示(依赖运行时剖析)能深入洞察性能瓶颈,但因开销大且固有非确定性,难以用于大规模任务。本文提出一种新型准动态程序表示框架,核心思想是建模程序的优化敏感性。引入程序行为谱(Program Behavior Spectrum),通过多样化优化序列探测程序中间表示,并量化其静态特征的变化。为有效编码高维连续谱,提出组合式学习方法:使用乘积量化(Product Quantization)将连续响应向量离散化为结构化的组合子词;随后训练多任务Transformer模型PQ-BERT,学习这些行为编码的深层上下文语法。在两个代表性编译优化任务——最佳优化阶段预测与-Oz优化收益预测——上的实验表明,该方法优于现有最优静态基线。代码已公开于https://github.com/Panhaolin2001/PREP/
原文摘要 · Abstract (English)
Learning effective numerical representations, or embeddings, of programs is a fundamental prerequisite for applying machine learning to automate and enhance compiler optimization. Prevailing paradigms, however, present a dilemma. Static representations, derived from source code or intermediate representation (IR), are efficient and deterministic but offer limited insight into how a program will behave or evolve under complex code transformations. Conversely, dynamic representations, which rely on runtime profiling, provide profound insights into performance bottlenecks but are often impractical for large-scale tasks due to prohibitive overhead and inherent non-determinism. This paper transcends this trade-off by proposing a novel quasi-dynamic framework for program representation. The core insight is to model a program's optimization sensitivity. We introduce the Program Behavior Spectrum, a new representation generated by probing a program's IR with a diverse set of optimization sequences and quantifying the resulting changes in its static features. To effectively encode this high-dimensional, continuous spectrum, we pioneer a compositional learning approach. Product Quantization is employed to discretize the continuous reaction vectors into structured, compositional sub-words. Subsequently, a multi-task Transformer model, termed PQ-BERT, is pre-trained to learn the deep contextual grammar of these behavioral codes. Comprehensive experiments on two representative compiler optimization tasks -- Best Pass Prediction and -Oz Benefit Prediction -- demonstrate that our method outperforms state-of-the-art static baselines. Our code is publicly available at https://github.com/Panhaolin2001/PREP/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。