arXiv:2505.11811cs.CL2025-05ACL被引 8

针对多跳问答,提出分类型智能推理框架,提升复杂问题解答效果。

BELLE: A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answering

  • 按问题类型匹配不同推理方法,将每种方法视为可调用的‘算子’。
  • 通过多智能体辩论生成最优推理计划,在多个数据集上显著超越基线。
  • 引入快慢思考机制监控观点变化合理性,适合复杂多跳问答场景使用。

多跳问答需查找多个相关段落并进行逐步推理以回答复杂问题。现有工作虽基于大语言模型从不同建模视角提出特定方法,但未考虑问题类型差异。本文深入分析公开多跳问答基准,将问题分为四类,并评估五种前沿方法:思维链(CoT)、单步、迭代步、子步与自适应步。发现不同类型问题对不同方法敏感度各异。为此,提出双层多智能体推理框架BELLE,聚焦问题类型与方法间的对应关系,通过提示不同方式使每类方法成为‘算子’。第一层包含多个智能体辩论,生成涵盖多种‘算子’的执行计划;第二层引入快速与慢速辩手,监控观点变化是否合理。大量实验表明,BELLE在多个数据集上显著优于强基线,且在更复杂的多跳问答中具有更高性价比。

原文摘要 · Abstract (English)

Multi-hop question answering (QA) involves finding multiple relevant passages and performing step-by-step reasoning to answer complex questions. Previous works on multi-hop QA employ specific methods from different modeling perspectives based on large language models (LLMs), regardless of the question types. In this paper, we first conduct an in-depth analysis of public multi-hop QA benchmarks, dividing the questions into four types and evaluating five types of cutting-edge methods for multi-hop QA: Chain-of-Thought (CoT), Single-step, Iterative-step, Sub-step, and Adaptive-step. We find that different types of multi-hop questions have varying degrees of sensitivity to different types of methods. Thus, we propose a Bi-levEL muLti-agEnt reasoning (BELLE) framework to address multi-hop QA by specifically focusing on the correspondence between question types and methods, where each type of method is regarded as an ''operator'' by prompting LLMs differently. The first level of BELLE includes multiple agents that debate to obtain an executive plan of combined ''operators'' to address the multi-hop QA task comprehensively. During the debate, in addition to the basic roles of affirmative debater, negative debater, and judge, at the second level, we further leverage fast and slow debaters to monitor whether changes in viewpoints are reasonable. Extensive experiments demonstrate that BELLE significantly outperforms strong baselines in various datasets. Additionally, the model consumption of BELLE is higher cost-effectiveness than that of single models in more complex multi-hop QA scenarios.

多跳问答智能体协作推理框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。