用LLM生成代码嵌入，显著提升程序分析与优化效果
LLM-Based Embeddings for Program Analysis and Optimization
- 用LLM对源码和中间表示代码分块嵌入后聚合
- 算法分类错误率降至1.54%，比当前最优提升12%
- 适合关注代码优化与AI辅助编译的开发者
近期研究揭示了机器学习，特别是大语言模型（LLMs）在程序分析与优化中的潜力。我们首次将基于LLMCompiler——一个在中间表示（IR）代码上大规模预训练的LLM——生成的程序嵌入，应用于典型的程序分析与优化任务。通过简单方法：将程序分块，用预训练的LLM独立嵌入每块，再聚合为整体程序嵌入。实验表明，结合源码与IR嵌入，在算法分类任务中实现1.54%的错误率，较当前最优水平提升12%，并在异构设备映射任务上达到有竞争力的准确率。结果表明，训练具备性能感知能力的LLM来嵌入IR代码，可能在代码优化任务中取得最先进成果。
原文摘要 · Abstract (English)
Recent advances have highlighted the potential of machine learning, particularly Large Language Models (LLMs), for analyzing and optimizing programs. We present the first application of program embeddings from LLMCompiler---an LLM massively pretrained on intermediate representation (IR) code---to representative program analysis and optimization tasks. We generate program embeddings directly from source and IR code using a simple approach: split programs into chunks, independently embed each chunk with pretrained LLMs, and then aggregate the chunk embeddings into a single program embedding. Our experiments show that combining source and IR code embeddings achieves an error rate of 1.54\% in algorithm classification, a 12\% improvement over the current state-of-the-art, and a competitive accuracy on heterogeneous device mapping. These findings suggest that training a performance-aware LLM for embedding IR code might yield state-of-the-art results in code optimization tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。