用双模型辩论提升因果推理准确率,尤其改善反事实判断。
CRAwDAD: Causal Reasoning Augmentation with Dual-Agent Debate
- 让两个模型互相辩论,一个提因果推断,一个挑逻辑漏洞。
- 使强模型(DeepSeek-R1)整体准确率从78.03%升至87.45%。
- 适合需要高可信度因果分析的研究者与应用开发者。
人类在进行因果推理时,常需对比多个‘如果……会怎样’的假设才能确定最优解释。类似地,具备因果推理能力的语言模型可通过考虑多种干预和反事实来评估因果陈述的有效性。关键在于,这种推理更像是一种内在对话而非单一计算。本文提出一种双代理辩论框架:一个模型提供结构化因果推理,另一个则批判性审视其逻辑缺陷。当意见分歧时,双方尝试说服对方,挑战彼此逻辑并不断修正结论,直至达成一致。为利用这一思辨过程,我们采用推理型语言模型,其在因果推理与对抗辩论方面的能力尚未被充分挖掘。我们在CLadder数据集上评估该方法,该数据集将自然语言问题与形式化因果图对应,覆盖Pearl因果阶梯的全部三阶。使用Qwen3和DeepSeek-R1作为辩论代理,结果显示:多智能体辩论使DeepSeek-R1的整体准确率从78.03%提升至87.45%,其中反事实类准确率从67.94%提升至80.04%;同样,Qwen3的整体准确率由84.16%升至89.41%,反事实类由71.53%升至80.35%。表明即使强模型也能从与弱模型的辩论中获益显著。结果凸显推理模型作为多智能体系统构建模块在因果推理中的潜力,并强调多元视角对解决因果问题的重要性。
原文摘要 · Abstract (English)
When people reason about cause and effect, they often consider many competing "what if" scenarios before deciding which explanation fits best. Analogously, advanced language models capable of causal inference can consider multiple interventions and counterfactuals to judge the validity of causal claims. Crucially, this type of reasoning is less like a single calculation and more like an internal dialogue between alternative hypotheses. In this paper, we make this dialogue explicit through a dual-agent debate framework where one model provides a structured causal inference, and the other critically examines this reasoning for logical flaws. When disagreements arise, the agents attempt to persuade each other, challenging each other's logic and revising their conclusions until they converge on a mutually agreed answer. To take advantage of this deliberative process, we specifically use reasoning language models, whose strengths in both causal inference and adversarial debate remain under-explored relative to standard large language models. We evaluate our approach on the CLadder dataset, a benchmark linking natural language questions to formally defined causal graphs across all three rungs of Pearl's ladder of causation. With Qwen3 and DeepSeek-R1 as debater agents, we demonstrate that multi-agent debate improves DeepSeek-R1's overall accuracy in causal inference from 78.03% to 87.45%, with the counterfactual category specifically improving from 67.94% to 80.04% accuracy. Similarly, Qwen3's overall accuracy improves from 84.16% to 89.41%, and counterfactual questions from 71.53% to 80.35%, showing that even strong models can still benefit greatly from debate with weaker agents. Our results highlight the potential of reasoning models as building blocks for multi-agent systems in causal inference, and demonstrate the importance of diverse perspectives in causal problem-solving.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。