研究小模型在自动修复代码问题框架中的能效表现，发现当前架构浪费大量能源。
SWEnergy: An Empirical Study on Energy Efficiency in Agentic Issue Resolution Frameworks with SLMs
- 对比四种框架在小模型下的运行效率，聚焦能效与资源消耗
- 最耗能框架比最省能高出9.4倍，但任务成功率接近零
- 框架设计缺陷导致能效低下，需主动应对小模型短板
基于大语言模型的自主智能体在软件工程中依赖大型专有模型，限制了本地部署。这推动了小型语言模型（SLMs）的发展，但其在复杂自动化问题解决框架中的实际效果和效率仍不明确。本研究系统评估了四个主流智能体框架（SWE-Agent、OpenHands、Mini SWE Agent、AutoCodeRover）在使用两个SLMs（Gemma-3 4B、Qwen-3 1.7B）时的表现，基于SWE-bench Verified Mini基准，在固定硬件上对150次运行进行能量、持续时间、令牌使用量和内存消耗测量。结果表明，框架架构是能耗的主要决定因素：AutoCodeRover（Gemma）平均能耗为OpenHands（Gemma）的9.4倍；然而，这些高能耗并未带来有效产出，任务解决率接近零，说明当前框架在搭配小模型时产生大量无用推理循环。小模型的有限推理能力是成功瓶颈，而框架设计则成为效率瓶颈。结论指出，面向大模型设计的框架在小模型上效率极低，亟需从被动调度转向主动管理小模型弱点的新型架构。
原文摘要 · Abstract (English)
Context. LLM-based autonomous agents in software engineering rely on large, proprietary models, limiting local deployment. This has spurred interest in Small Language Models (SLMs), but their practical effectiveness and efficiency within complex agentic frameworks for automated issue resolution remain poorly understood. Goal. We investigate the performance, energy efficiency, and resource consumption of four leading agentic issue resolution frameworks when deliberately constrained to using SLMs. We aim to assess the viability of these systems for this task in resource-limited settings and characterize the resulting trade-offs. Method. We conduct a controlled evaluation of four leading agentic frameworks (SWE-Agent, OpenHands, Mini SWE Agent, AutoCodeRover) using two SLMs (Gemma-3 4B, Qwen-3 1.7B) on the SWE-bench Verified Mini benchmark. On fixed hardware, we measure energy, duration, token usage, and memory over 150 runs per configuration. Results. We find that framework architecture is the primary driver of energy consumption. The most energy-intensive framework, AutoCodeRover (Gemma), consumed 9.4x more energy on average than the least energy-intensive, OpenHands (Gemma). However, this energy is largely wasted. Task resolution rates were near-zero, demonstrating that current frameworks, when paired with SLMs, consume significant energy on unproductive reasoning loops. The SLM's limited reasoning was the bottleneck for success, but the framework's design was the bottleneck for efficiency. Conclusions. Current agentic frameworks, designed for powerful LLMs, fail to operate efficiently with SLMs. We find that framework architecture is the primary driver of energy consumption, but this energy is largely wasted due to the SLMs' limited reasoning. Viable low-energy solutions require shifting from passive orchestration to architectures that actively manage SLM weaknesses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。