提出多智能体框架ReMind,提升大模型代码推理能力。
\texttt{ReMind}: Understanding Deductive Code Reasoning in LLMs
- 用三个智能体协同:改写代码、追踪变量、检查逻辑漏洞
- 在两个基准上使大模型零样本推理准确率显著提升
- 适合研究大模型代码理解与自动验证的开发者
大型语言模型(LLMs)在代码相关任务中取得显著进展,但在推理程序执行过程的演绎式代码推理方面仍存在明显不足。本文通过实证研究发现三大核心挑战:生成与推理能力间存在内在差距;对源代码存在持续偏倚;在复杂基准上零样本泛化能力弱。为此,提出多智能体框架ReMind,包含Mutator(生成代码变体以缓解源代码偏倚)、Executor(逐步追踪变量状态暴露不一致)、Inspector(识别错误推理步骤并优化控制流)。三者协同可系统性发现并修正推理缺陷。在两个基准和五种大模型上的实验表明,ReMind显著优于基线方法,在演绎式代码推理任务中实现卓越性能与稳健的零样本泛化能力。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have achieved remarkable progress in code-related tasks. Despite their advancement, empirical evidence reveals that they still struggle with \emph{deductive code reasoning}, the ability to reason about the program execution process. While prior studies have recognized this limitation, the underlying causes remain largely underexplored. In this paper, we begin by presenting a comprehensive empirical study that reveals three key challenges undermining deductive code reasoning: (1) an intrinsic gap between generation and reasoning abilities, (2) a consistent bias towards code sources, and (3) weak zero-shot generalization on complex benchmarks. In light of these challenges, we propose \texttt{ReMind}, a multi-agent framework composed of \texttt{Mutator}, \texttt{Executor}, and \texttt{Inspector}. The \texttt{Mutator} generates code variants to mitigate bias towards code sources, the \texttt{Executor} traces variable states step-by-step to expose inconsistency, and the \texttt{Inspector} identifies problematic reasoning steps and provides control-flow refinement to bridge the intrinsic reasoning gap. Through their coordinated collaboration, \texttt{ReMind} systematically identifies and refines reasoning flaws, achieving outstanding performance and enabling robust zero-shot generalization. Extensive experiments on two benchmarks with five LLMs demonstrate the superior advantages of \texttt{ReMind} compared to baseline approaches in deductive code reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。