用强化学习动态选最精简的证据,让长文本问答更准
Context-Picker: Dynamic context selection using multi-stage reinforcement learning
- 分两阶段强化学习,先抓关键信息再剔除冗余
- 在5个数据集上超越主流RAG方法,准确率提升显著
- 适合需要精准推理的长文本问答场景
在长上下文问答任务中,如何动态选择恰当的上下文范围仍是核心难题。上下文不足会遗漏关键信息,过多则引入噪声并降低答案质量。传统方法如固定数量召回或重排序难以自适应调整。针对事实类问题通常仅依赖少数精确证据的特点,本文提出Context-Picker——一种感知推理的框架,将上下文选择重构为寻找最小充分证据子集的任务。该方法采用类人化的两阶段强化学习:第一阶段提升关键段落的召回率,第二阶段聚焦去除冗余以提炼紧凑证据集。为解决奖励稀疏问题,设计离线证据蒸馏流程,通过留一法(LOO)挖掘“最小充分集合”,提供密集且任务对齐的监督信号。在五个长上下文与多跳问答数据集上的实验表明,本方法优于强基线RAG模型,并实现更高答案准确率。消融研究显示,粗到精优化策略、冗余感知奖励设计以及策略生成的推理路径均对性能提升有显著贡献。
原文摘要 · Abstract (English)
In long-context question answering, selecting the appropriate scope of context for a query remains a key and unresolved challenge. Insufficient context can lead to missing essential information, whereas excessive context often introduces noise and degrades answer quality. Conventional methods, such as retrieving a fixed number of passages or applying reranking, struggle to dynamically determine which context to include. This is especially problematic for factoid questions, which typically depend only on a few precise pieces of evidence. To overcome this limitation, we propose Context-Picker, a reasoning-aware framework that reframes context selection as the task of identifying a minimal sufficient evidence subset, moving beyond conventional similarity-based ranking. Context-Picker uses a human-inspired two-stage reinforcement learning schedule: stage 1 focuses on improving the recall rate of critical passages, and stage 2 prioritizes pruning redundancy to distill a compact evidence set. To resolve reward sparsity, we propose an offline evidence distillation pipeline that mines ``minimal sufficient sets" via a Leave-One-Out (LOO) procedure, providing dense and task-aligned supervision. Experiments on five long-context and multi-hop QA datasets demonstrate that our method outperforms strong RAG baselines and achieved higher answer accuracy. Ablation studies also indicate that our coarse-to-fine optimization schedule, the redundancy-aware reward shaping, along with the rationale generated by the policy, all contribute substantially to these gains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。