arXiv:2510.07958cs.CLcs.AI2025-10被引 4

让AI理解问题的多种正确答案,提升问答系统可靠性。

A$^2$Search: Ambiguity-Aware Question Answering with Reinforcement Learning

  • 用强化学习自动识别模糊问题并收集多答案
  • 单次推理下多跳问答平均答对率48.4%,超越更大模型
  • 无需人工标注,适合复杂开放域问答任务

大语言模型与强化学习的进步推动了开放域问答的发展,但现有模型仍难以应对存在多个合理答案的问题。传统基准通常假设唯一标准答案,导致训练信号失真。现有解决方法依赖昂贵的人工标注,难以扩展至多跳数据集如HotpotQA和MuSiQue。本文提出A²Search,一种无标注、端到端的模糊性处理框架。其核心为自动化流程:通过轨迹采样与证据验证检测模糊问题并收集备选答案;再利用精心设计的AnsF1奖励进行强化学习优化,自然支持多答案。在八个开放域问答基准上的实验表明,A²Search达到新基准性能。仅一次推理,A²Search-7B在四个多跳基准上平均AnsF1@1达48.4%,优于更大模型ReSearch-32B(46.2%)。大量分析显示,该方法有效化解模糊性且跨基准泛化能力强,证明拥抱模糊性对构建可靠问答系统至关重要。代码、数据及模型权重见https://github.com/zfj1998/A2Search。

原文摘要 · Abstract (English)

Recent advances in Large Language Models (LLMs) and Reinforcement Learning (RL) have led to strong performance in open-domain question answering (QA). However, existing models still struggle with questions that admit multiple valid answers. Standard QA benchmarks, which typically assume a single gold answer, overlook this reality and thus produce inappropriate training signals. Existing attempts to handle ambiguity often rely on costly manual annotation, which is difficult to scale to multi-hop datasets such as HotpotQA and MuSiQue. In this paper, we present A$^2$Search, an annotation-free, end-to-end training framework to recognize and handle ambiguity. At its core is an automated pipeline that detects ambiguous questions and gathers alternative answers via trajectory sampling and evidence verification. The model is then optimized with RL using a carefully designed $\mathrm{AnsF1}$ reward, which naturally accommodates multiple answers. Experiments on eight open-domain QA benchmarks demonstrate that A$^2$Search achieves new state-of-the-art performance. With only a single rollout, A$^2$Search-7B yields an average $\mathrm{AnsF1}@1$ score of $48.4\%$ across four multi-hop benchmarks, outperforming all strong baselines, including the substantially larger ReSearch-32B ($46.2\%$). Extensive analyses further show that A$^2$Search resolves ambiguity and generalizes across benchmarks, highlighting that embracing ambiguity is essential for building more reliable QA systems. Our code, data, and model weights can be found at https://github.com/zfj1998/A2Search

问答系统强化学习模糊性处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。