用统一语言模型直接预测代码的内存、延迟和精度，无需复杂特征工程。
Regression Language Models for Code
- 基于冻结的LLM编码器构建统一回归模型，直接从代码文本预测性能指标。
- 300M参数模型在APPS竞赛数据上达0.9以上斯皮尔曼相关性，跨24语言平均超0.5。
- 可替代图神经网络预测神经架构搜索性能，支持多硬件平台延迟预测。
我们研究代码到度量的回归任务：直接预测代码执行的数值结果，该任务因编程语言的开放性而极具挑战。以往方法依赖繁重且领域特定的特征工程，本文提出一种统一的回归语言模型（RLM），仅使用冻结的LLM编码器即可同时预测多种代码的性能：(i) 跨Python、C++等多门高级语言的内存占用；(ii) Triton GPU内核的执行延迟；(iii) ONNX格式表示的神经网络的准确率与速度。特别地，基于T5Gemma的300M参数小型RLM在APPS竞赛提交代码上获得超过0.9的斯皮尔曼等级相关性；单一模型在CodeNet提供的24种编程语言上实现平均0.5以上的斯皮尔曼相关性。此外，该模型在五个经典神经架构搜索设计空间上取得0.46的最高平均肯德尔τ相关性，显著超越此前依赖图神经网络的方法，并能同时预测多种硬件平台上的架构延迟。
原文摘要 · Abstract (English)
We study code-to-metric regression: predicting numeric outcomes of code executions, a challenging task due to the open-ended nature of programming languages. While prior methods have resorted to heavy and domain-specific feature engineering, we show that a single unified Regression Language Model (RLM) using a frozen LLM encoder can simultaneously predict directly from text, (i) the memory footprint of code across multiple high-level languages such as Python and C++, (ii) the latency of Triton GPU kernels, and (iii) the accuracy and speed of trained neural networks represented in ONNX. In particular, a relatively small 300M parameter RLM based on T5Gemma, obtains >0.9 Spearman-rank on competitive programming submissions from APPS, and a single unified model achieves >0.5 average Spearman-rank across 24 different programming languages from CodeNet. Furthermore, the RLM can obtain the highest average Kendall-Tau of 0.46 on five classic NAS design spaces previously dominated by graph neural networks, and simultaneously predict architecture latencies on numerous hardware platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。