用心理学启发的双流架构提升大模型代码调试效率与准确率
Dual-Process Scaffold Reasoning for Enhancing LLM Code Debugging
- 构建三流协同框架：支架流、分析流、整合流，模拟人类思维过程
- 在DebugBench上达88.91%通过率，单题推理仅需5.36秒
- 适用于需要高精度与高效推理的编程辅助场景
近期大语言模型在各类基准测试中展现出复杂问题求解能力，但如何平衡推理步骤的复杂性与计算效率仍是未解难题。受心理学理论启发，将模型输出视为系统1，中间推理步骤视为系统2，但对系统2的深入研究仍不足。为此，本文提出一种基于心理学启发的支架式推理框架（Scaffold Reasoning），包含支架流、分析流和整合流。支架流生成参考代码，分析流处理有缺陷代码，整合流将二者融合。该框架在DebugBench上实现88.91%的通过率，平均每个问题推理时间为5.36秒，优于多种主流大模型的推理方法。进一步分析揭示了不同认知路径在不同难度与错误类型下的优劣。结果表明，该框架与人类认知过程高度一致。
原文摘要 · Abstract (English)
Recent LLMs have demonstrated sophisticated problem-solving capabilities on various benchmarks through advanced reasoning algorithms. However, the key research question of identifying reasoning steps that balance complexity and computational efficiency remains unsolved. Recent research has increasingly drawn upon psychological theories to explore strategies for optimizing cognitive pathways. The LLM's final outputs and intermediate steps are regarded as System 1 and System 2, respectively. However, an in-depth exploration of the System 2 reasoning is still lacking. Therefore, we propose a novel psychologically backed Scaffold Reasoning framework for code debugging, which encompasses the Scaffold Stream, Analytic Stream, and Integration Stream. The construction of reference code within the Scaffold Stream is integrated with the buggy code analysis results produced by the Analytic Stream through the Integration Stream. Our framework achieves an 88.91% pass rate and an average inference time of 5.36 seconds per-problem on DebugBench, outperforming other reasoning approaches across various LLMs in both reasoning accuracy and efficiency. Further analyses elucidate the advantages and limitations of various cognitive pathways across varying problem difficulties and bug types. Our findings also corroborate the alignment of the proposed Scaffold Reasoning framework with human cognitive processes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。