用双检测多智能体系统提升论文与代码差异检测的准确率
Dude: A Dual-Detection Multi-Agent System for Paper-Code Discrepancy Detection
- 设计双检测机制，对论文和代码进行分层对比
- 在真实数据集上提升召回率22.8%，F1分数提高18.7%
- 适合科研评审、开源项目质量检查场景
大语言模型赋能的论文-代码差异检测因研究投稿量激增而受到关注。然而，现有单智能体方法受限于上下文容量和片面检测，导致差异召回率偏低。本文提出Dude，首个用于论文-代码差异检测的双检测多智能体系统。我们发现论文语言与代码语言在粒度上的不对称性会引发多智能体系统中的过度解释与误报问题，增加假阳性。为此，Dude引入粒度对齐协商机制与两阶段显著性过滤，有效避免误报。在真实世界论文-代码差异数据集上的实验表明，相比基线方法，Dude在召回率上最高提升22.8%，F1分数最高提升18.7%。
原文摘要 · Abstract (English)
LLM-empowered paper-code discrepancy detection has received growing concern since the scaling of research submissions exceeds the manual review capability. However, the limited context capacity and one-sided discrepancy detection of existing single-agent LLM paradigms lead to an inferior recall performance in detecting discrepancies. In this paper, we propose Dude, the first Dual-Detection Multi-Agent System for paper-code discrepancy detection. We discover that the granularity asymmetry of the paper-language and code-language introduces over-interpretation and over-reporting challenges in a multi-agent system design for discrepancy detection, resulting in increasing false positives. To address this, we propose a granularity-aligned negotiation and a two-stage salience-filtering mechanism in Dude, which effectively prevents agents from falsely reporting discrepancies. Experimental results in real-world paper-code discrepancy datasets showcase Dude's significant recall and precision improvement by up to 22.8%, increasing F1 score by up to 18.7% compared to baseline methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。