用多个小模型投票提升缺陷定位准确率，兼顾效率与成本
COSMosFL: Ensemble of Small Language Models for Fault Localisation
- 通过小模型投票构建集成系统，平衡性能与资源消耗
- 在Defects4J上实现精度与推理成本的帕累托最优
- 适合关注低耗高效代码分析的开发者和研究者
大型语言模型正被广泛用于软件工程工具开发，但其依赖封闭的大模型带来安全、成本与环境问题。近年来开源小型语言模型（SLMs）兴起，虽更节能易部署，但性能普遍弱于大模型。本文提出COSMos，一种基于任务级集成的投票机制，将基于LLM的缺陷定位技术AutoFL与SLMs结合。在Defects4J数据集上的实证评估显示，COSMos可构建有效集成模型，在缺陷定位准确率与推理成本（能耗、时间、令牌数）之间实现帕累托最优，显著优于单个模型。
原文摘要 · Abstract (English)
LLMs are rapidly being adopted to build powerful tools and agents for software engineering, but most of them rely heavily on extremely large closed-source models. This, in turn, can hinder wider adoption due to security issues as well as financial cost and environmental impact. Recently, a number of open source Small Language Models (SLMs) are being released and gaining traction. While SLMs are smaller, more energy-efficient, and therefore easier to locally deploy, they tend to show worse performance when compared to larger closed LLMs. We present COSMos, a task-level LLM ensemble technique that uses voting mechanism, to provide a broader range of choice between SLMs and LLMs. We instantiate COSMos with an LLM-based Fault Localisation technique, AutoFL, and report the cost-benefit trade-off between LLM accuracy and various costs such as energy consumption, inference time, and the number of tokens used. An empirical evaluation using Defects4J shows that COSMos can build effective ensembles that can achieve Pareto-optimality in terms of FL accuracy and inference cost, when compared to individual models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。