让大模型优化代码更贴近真实部署，显著降低推理延迟。
LLM4LLM: Bridging Kernel Benchmarks and Real Deployment via Closed-Loop Agentic Optimization
- 用闭环智能体从真实推理脚本中提取优化任务
- 在A100/H100上实现3.91×/6.98×几何均值加速
- 适合关注实际部署性能的深度学习系统开发者
大语言模型在低级代码和核函数优化方面能力不断增强，但孤立的内核基准测试仅能代理语言模型推理中的实际部署行为。我们发现基准测试与真实部署之间存在差距：在独立测试中表现正确且快速的候选内核，在集成到真实推理工作负载后可能表现出不同的性能、安全或阶段行为。为此，我们提出LLM4LLM，一个面向部署的闭环优化框架，从目标推理脚本出发，提取阶段感知的优化任务，通过经验引导的周期性智能体进行搜索，并以模型内验证接受补丁。在A100和H100 GPU上的十个语言模型推理工作负载上，LLM4LLM对每个评估模型均提升了端到端延迟，实现了3.91×/6.98×的几何平均加速；作为内核级证据，其在KernelBench Level 2上也达到了最高2.745×的几何平均加速。
原文摘要 · Abstract (English)
Large language models have become increasingly capable agents for low-level code and kernel optimization, but isolated kernel benchmarks provide only a proxy for the deployment behavior that matters in language-model inference. We identify a benchmark-to-deployment gap: candidate kernels that appear correct and fast in standalone harnesses can exhibit different performance, safety, or phase behavior after integration into a real inference workload. We introduce LLM4LLM, a deployment-aware closed-loop optimization framework that starts from a target inference script, extracts phase-aware optimization tasks, searches with an experience-guided episodic agent, and accepts patches through in-model validation. Across ten language-model inference workloads on A100 and H100 GPUs, LLM4LLM improves end-to-end latency for every evaluated model, achieving 3.91$\times$/6.98$\times$ geometric-mean speedups on A100/H100; as supporting kernel-level evidence, it also attains up to 2.745$\times$ GeoMean speedup on KernelBench Level 2.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。