FASE快速评估代码质量，比现有方法更准且省算力。
FASE: Fast Adaptive Semantic Entropy for Code Quality
- 基于结构与语义图的最小生成树，近似计算代码功能正确性。
- 在HumanEval上提升25%相关性，ROCAUC提高19%，接近真实测试结果。
- 计算开销仅传统方法的0.3%，适合实际多智能体开发流程。
多智能体代码生成通过模拟人类软件工程生命周期，为自主软件开发提供了新范式。然而，大模型幻觉和智能体间错误传播严重制约系统可靠性。尽管语义熵能无真值情况下量化不确定性，但现有方法依赖昂贵的大模型等价性验证。本文提出快速自适应语义熵（FASE），基于结构与语义差异图的最小生成树近似功能正确性。在HumanEval与BigCodeBench上的实验表明，相较于基于大模型蕴含的语义熵，FASE在Qwen3-Embedding-8B模型下平均提升25%的斯皮尔曼相关性，Pass@1的ROCAUC得分提高19%。同时，由于摒弃了耗时的大模型等价性评估，FASE计算开销仅为传统方法的约0.3%。结果表明，FASE是实现高效不确定量化的真实多智能体工作流中的实用解决方案。
原文摘要 · Abstract (English)
Multi-agent code generation offers a promising paradigm for autonomous software development by simulating the human software engineering lifecycle. However, system reliability remains hindered by LLM hallucinations and error propagation across interacting agents. While semantic entropy provides a principled way to quantify uncertainty without ground-truth answers, current methods often rely on costly LLM-driven equivalence checks. In this work, we introduce Fast Adaptive Semantic Entropy (FASE), a novel metric that approximates functional correctness based on the minimum spanning tree of structural and semantic dissimilarity graphs. Evaluations on HumanEval and BigCodeBench demonstrate that FASE outperforms state-of-the-art semantic entropy by LLM entailment, achieving a 25% average improvement in Spearman correlation and a 19% increase in ROCAUC score against Pass@1 from ground-truth test cases when using the Qwen3-Embedding-8B model. Furthermore, by eliminating costly LLM-driven equivalence evaluation, FASE incurs negligible computational overhead, requiring only approximately 0.3% of the runtime cost of traditional semantic entropy approaches. These results position FASE as a practical, cost-effective solution for optimizing uncertainty quantification in real-world multi-agent workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。