让模型学会判断段落间关联,提升多跳问答的准确性
Modeling Contextual Passage Utility for Multihop Question Answering
- 用小规模Transformer模型捕捉段落间的上下文依赖关系
- 基于推理轨迹生成训练数据,提升段落实用度预测精度
- 适合需要精准信息融合的多跳问答场景
多跳问答需从多个文本段落中识别并整合信息。现有检索方法多关注相关性,但忽略段落的实际效用——某些段落可能冗余,反而引入噪声与错误。现有方法独立评估段落效用,未考虑其在多跳推理中的上下文依赖:一个段落的价值取决于它与其他段落的关系,如是否提供互补信息或构成关键连接。本文提出一种轻量级方法,建模段落间的上下文效用,通过微调小型Transformer模型预测多跳问答中段落的效用得分。利用先进推理模型的推理轨迹,获取段落使用顺序,构建合成训练数据。大量实验表明,基于效用的段落重排序显著优于基于相关性的方法,提升了下游问答性能。
原文摘要 · Abstract (English)
Multihop Question Answering (QA) requires systems to identify and synthesize information from multiple text passages. While most prior retrieval methods assist in identifying relevant passages for QA, further assessing the utility of the passages can help in removing redundant ones, which may otherwise add to noise and inaccuracies in the generated answers. Existing utility prediction approaches model passage utility independently, overlooking a critical aspect of multihop reasoning: the utility of a passage can be context-dependent, influenced by its relation to other passages - whether it provides complementary information or forms a crucial link in conjunction with others. In this paper, we propose a lightweight approach to model contextual passage utility, accounting for inter-passage dependencies. We fine-tune a small transformer-based model to predict passage utility scores for multihop QA. We leverage the reasoning traces from an advanced reasoning model to capture the order in which passages are used to answer a question and obtain synthetic training data. Through comprehensive experiments, we demonstrate that our utility-based scoring of retrieved passages leads to improved reranking and downstream QA performance compared to relevance-based reranking methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。