用元提示自动适配多模型代码优化,提升工业级效率
Tuning LLM-based Code Optimization via Meta-Prompting: An Industrial Perspective
- 通过元提示动态融合项目信息生成跨模型优化提示
- 在5个真实代码库上实现最高19.06%性能提升
- 96%优质优化来自有意义的代码修改,适合工业部署者
当前利用多个大语言模型(LLMs)进行自动化代码优化日益流行,但工业平台在部署多模型系统时面临关键挑战:为某一模型优化的提示在其他模型上常失效,需高昂代价进行模型专属提示工程。这一跨模型提示瓶颈严重限制了多模型系统在生产环境中的实际应用。本文提出元提示代码优化框架MPCO,可自动在多种LLMs上生成高质量、任务特定的提示,同时满足工业效率要求。MPCO通过元提示动态合成上下文感知的优化提示,整合项目元数据、任务需求和模型特异性上下文。该框架是ARTEMIS代码优化平台中自动化验证与扩展的核心组件。我们在五个真实代码库上进行了全面评估,累计366小时运行时基准测试显示,与基线方法相比,MPCO实现了最高达19.06%的整体性能提升,且在所有系统中表现最佳统计排名。分析表明,96%的顶级优化源于有意义的代码修改。通过系统的消融实验和元提示器敏感性分析,我们确认全面上下文整合对有效元提示至关重要,并发现主流大模型可作为有效的元提示器,为工业实践提供可操作洞见。
原文摘要 · Abstract (English)
There is a growing interest in leveraging multiple large language models (LLMs) for automated code optimization. However, industrial platforms deploying multiple LLMs face a critical challenge: prompts optimized for one LLM often fail with others, requiring expensive model-specific prompt engineering. This cross-model prompt engineering bottleneck severely limits the practical deployment of multi-LLM systems in production environments. We introduce Meta-Prompted Code Optimization (MPCO), a framework that automatically generates high-quality, task-specific prompts across diverse LLMs while maintaining industrial efficiency requirements. MPCO leverages metaprompting to dynamically synthesize context-aware optimization prompts by integrating project metadata, task requirements, and LLM-specific contexts. It is an essential part of the ARTEMIS code optimization platform for automated validation and scaling. Our comprehensive evaluation on five real-world codebases with 366 hours of runtime benchmarking demonstrates MPCO's effectiveness: it achieves overall performance improvements up to 19.06% with the best statistical rank across all systems compared to baseline methods. Analysis shows that 96% of the top-performing optimizations stem from meaningful edits. Through systematic ablation studies and meta-prompter sensitivity analysis, we identify that comprehensive context integration is essential for effective meta-prompting and that major LLMs can serve effectively as meta-prompters, providing actionable insights for industrial practitioners.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。