用信息增益奖励改进大模型知识问答的搜索效率
MCTS-KBQA: Monte Carlo Tree Search with Information Gain Rewards for Knowledge Base Question Answering
- 用信息增益替代终点奖励，实现中间状态即时反馈
- 在四个基准上准确率优于线性基线，且更省计算资源
- 无需额外训练奖励模型，直接用开源大模型生成奖励
本文研究如何通过蒙特卡洛树搜索（MCTS）提升大语言模型（LLM）在知识库问答（KBQA）中的推理能力。将MCTS应用于基于LLM的KBQA仍具挑战：奖励设计困难且基于回滚的搜索计算开销大。现有方法或依赖直接的LLM打分，或需大量数据训练独立奖励模型，且通常仅在终止状态提供奖励。为此，我们提出Fast MCTS，以信息增益（IG）奖励替代终端回滚，用于中间状态。该奖励通过问题条件化的PPL比率代理实现，基于净化后的交互历史，由开源指令型LLM前向传播计算得出，无需额外奖励模型训练。在四个KBQA基准上的实验表明，Fast MCTS始终优于线性基线，并在准确率-成本权衡上普遍优于基于回滚的经典MCTS。代码与数据见https://github.com/JimXiongGM/MCTS-KBQA。
原文摘要 · Abstract (English)
This work investigates how to improve large language model (LLM)-based reasoning for knowledge base question answering (KBQA) via Monte Carlo Tree Search (MCTS). Applying MCTS to LLM-based KBQA remains challenging because reward design is difficult and rollout-based search is computationally expensive. Existing MCTS-style methods either rely on direct LLM scoring or require substantial data to train separate reward models, and they often provide rewards only at terminal states. To address these limitations, we propose Fast MCTS, which replaces terminal rollouts with an information gain (IG) reward for intermediate states. The IG reward is implemented as a question-conditioned PPL-ratio proxy over sanitized interaction histories, computed by forward passes of an open-source instruction LLM without additional reward-model training. Experiments on four KBQA benchmarks show that Fast MCTS consistently outperforms linear baselines and generally improves the accuracy-cost trade-off relative to rollout-based Classic MCTS. Code and data are available at https://github.com/JimXiongGM/MCTS-KBQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。