arXiv:2510.09049cs.AIcs.SE2025-10

用专家协作辩论提升代码时间复杂度预测准确率

MEC$^3$O: Multi-Expert Consensus for Code Time Complexity Prediction

  • 让不同大模型专攻擅长的复杂度类别,形成专家团队
  • 在CodeComplex数据集上比基线高至少10%准确率和宏F1
  • 无需额外裁判模型,适合需要高精度复杂度分析的开发者

预测源代码复杂度对软件开发和算法分析至关重要。近期研究发现,未经微调的大语言模型在某些复杂度类别上表现不佳,表明单一模型无法胜任所有类别。为此,本文提出MEC$^3$O,一种多专家共识系统,扩展了多代理辩论框架。该方法根据各模型性能将大模型分配至特定复杂度类别,并提供针对性指令,使其成为领域专家。这些专家进行结构化辩论,最终通过加权共识机制融合预测结果。该策略有效缓解思维退化问题,避免依赖独立裁判模型,防止错误多数意见收敛。在CodeComplex数据集上的实验表明,MEC$^3$O优于开源基线,准确率与宏F1均提升至少10%;其宏F1平均超过GPT-4o-mini,且与GPT-4o及GPT-o4-mini持平。结果验证了多专家辩论与加权共识策略的有效性。代码与数据已公开于https://github.com/suhanmen/MECO。

原文摘要 · Abstract (English)

Predicting the complexity of source code is essential for software development and algorithm analysis. Recently, Baik et al. (2025) introduced CodeComplex for code time complexity prediction. The paper shows that LLMs without fine-tuning struggle with certain complexity classes. This suggests that no single LLM excels at every class, but rather each model shows advantages in certain classes. We propose MEC$^3$O, a multi-expert consensus system, which extends the multi-agent debate frameworks. MEC$^3$O assigns LLMs to complexity classes based on their performance and provides them with class-specialized instructions, turning them into experts. These experts engage in structured debates, and their predictions are integrated through a weighted consensus mechanism. Our expertise assignments to LLMs effectively handle Degeneration-of-Thought, reducing reliance on a separate judge model, and preventing convergence to incorrect majority opinions. Experiments on CodeComplex show that MEC$^3$O outperforms the open-source baselines, achieving at least 10% higher accuracy and macro-F1 scores. It also surpasses GPT-4o-mini in macro-F1 scores on average and demonstrates competitive on-par F1 scores to GPT-4o and GPT-o4-mini on average. This demonstrates the effectiveness of multi-expert debates and weight consensus strategy to generate the final predictions. Our code and data is available at https://github.com/suhanmen/MECO.

代码分析大模型复杂度预测多专家

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。