用大模型预测OpenCL程序的GPU性能，误差可控。
LLMPerf: GPU Performance Modeling meets Large Language Models
- 用大语言模型直接估算OpenCL程序的运行时性能。
- 在自动生成数据集上误差为24.25%，真实程序上为46.1%。
- 首次探索大模型做性能建模，适合硬件优化与编译器研究者。
性能建模是程序成本分析的关键领域，当前依赖人工构建的模型，受限于程序与硬件条件，尤其在GPGPU复杂场景中表现不足。大语言模型（LLMs）在多种编程挑战中展现出高效性。本文首次将LLM引入性能建模，将其作为性能估计器。基于精心设计的大规模OpenCL数据集实验，揭示了使用LLM处理OpenCL设备源程序性能建模的潜力与主要困难。作为该方向的开创性研究，所提出的基于LLM的性能模型在大规模生成验证集上的平均绝对百分比误差为24.25%；在公开的OpenCL程序集上，误差为46.1%。
原文摘要 · Abstract (English)
Performance modeling, a pivotal domain in program cost analysis, currently relies on manually crafted models constrained by various program and hardware limitations, especially in the intricate landscape of GPGPU. Meanwhile, Large Language Models (LLMs) have demonstrated their effectiveness in addressing diverse programming challenges. Our work establishes a connection between LLMs and performance modeling, employing the LLM as a performance estimator. Through experimental exploration with carefully designed large-scale OpenCL datasets, we highlight the potential capability as well as the main difficulties of using LLMs in handling performance modeling tasks for OpenCL device source programs. As the first study for this line of work, our LLM-based performance model achieves a mean absolute percentage error of $24.25\%$ for a large-scale generated validation set. On a set of publicly available OpenCL programs, our model achieves a mean absolute percentage error of $46.1\%$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。