首次实证对比AI与人类在代码优化中的表现差异
How Do Agents Perform Code Optimization? An Empirical Study
- 对比324个AI与83个人类编写的优化提交
- AI优化代码缺少显式性能验证（45.7% vs 63.6%）
- AI与人类使用相似优化模式，但验证不足
性能优化是软件开发中关键且复杂的环节，需深入理解系统行为、算法权衡及精细的代码修改。尽管近年来人工智能编程代理在代码生成和缺陷修复方面取得进展，但其在真实性能优化任务中的表现仍不明确。本文首次对基于AIDev数据集的324个AI生成和83个人类编写的性能优化提交进行实证研究，分析了采纳率、可维护性、优化模式及验证实践。结果发现，相较于人类，AI生成的优化提交更少包含明确的性能验证（45.7% vs 63.6%，p=0.007）。此外，AI使用的优化模式与人类高度相似。文章进一步讨论了当前代理代码优化的局限与未来改进方向。
原文摘要 · Abstract (English)
Performance optimization is a critical yet challenging aspect of software development, often requiring a deep understanding of system behavior, algorithmic tradeoffs, and careful code modifications. Although recent advances in AI coding agents have accelerated code generation and bug fixing, little is known about how these agents perform on real-world performance optimization tasks. We present the first empirical study comparing agent- and human-authored performance optimization commits, analyzing 324 agent-generated and 83 human-authored PRs from the AIDev dataset across adoption, maintainability, optimization patterns, and validation practices. We find that AI-authored performance PRs are less likely to include explicit performance validation than human-authored PRs (45.7\% vs. 63.6\%, $p=0.007$). In addition, AI-authored PRs largely use the same optimization patterns as humans. We further discuss limitations and opportunities for advancing agentic code optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。