小模型让编译器自动并行化，性能最高提升43倍
Small Language Models as Compiler Experts: Auto-Parallelization for Heterogeneous Systems
- 用10亿参数小模型+多种推理策略生成并行代码
- 在11个真实算子上平均提速6.81倍，卷积最高达43.25倍
- 适合需要高效优化的科研与工业级异构计算场景
传统自动并行编译器依赖僵化启发式规则，在现代异构系统中表现受限。本文全面评估了三种约10亿参数的小语言模型（gemma3、llama3.2、qwen2.5）在编译器自动并行化中的表现，采用六种推理策略，测试11个来自科学计算、图算法和机器学习的真实内核。系统在376次评估中，相较LLVM Polly、TVM和Triton等强基准，平均获得6.81倍加速，卷积操作峰值达43.25倍。通过多维度分析验证了可扩展性，使用多种内存检测工具确保正确性，并在多种编译器与硬件平台上确认鲁棒性。结果表明，小型高效语言模型可作为复杂编译优化任务的强大推理引擎。
原文摘要 · Abstract (English)
Traditional auto-parallelizing compilers, reliant on rigid heuristics, struggle with the complexity of modern heterogeneous systems. This paper presents a comprehensive evaluation of small (approximately 1B parameter) language-model-driven compiler auto-parallelization. We evaluate three models: gemma3, llama3.2, and qwen2.5, using six reasoning strategies across 11 real-world kernels drawn from scientific computing, graph algorithms, and machine learning. Our system is benchmarked against strong compiler baselines, including LLVM Polly, TVM, and Triton. Across 376 total evaluations, the proposed approach achieves an average speedup of 6.81x and a peak performance of 43.25x on convolution operations. We analyze scalability, verify correctness using multiple sanitizers, and confirm robustness across diverse compilers and hardware platforms. Our results demonstrate that small, efficient language models can serve as powerful reasoning engines for complex compiler optimization tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。