arXiv:2504.05317cs.IRcs.AI2025-04ACL被引 4

用大模型生成带出处的问答数据,提升小模型问答可信度

On Synthesizing Data for Context Attribution in Question Answering

  • 用大模型自动生成带上下文依据的问答对
  • 合成数据让小模型在多任务中准确率提升显著
  • 适合需要可解释问答的落地场景

问答是大模型在真实场景中的主要应用之一,但其常产生虚假或误导性回答(即“幻觉”)。因此,将生成答案与提供上下文信息关联——即提供答案依据——对保障大模型可信性至关重要。该任务称为上下文归因。本文系统研究基于大模型的上下文归因方法,包括零样本推理、大模型集成及在由大模型生成的合成数据上微调小型语言模型。核心贡献是提出SynQA:一种生成式合成上下文归因数据的新策略。给定选定的上下文句,大模型生成由这些句子支持的问答对,利用大模型文本生成优势并确保合成数据中归因路径清晰。实验证明,通过SynQA生成的归因数据在不同问答任务和领域中对小型模型的微调极为有效。用户研究表明,经SynQA合成数据微调的小模型在实际问答中具有良好的上下文归因能力。

原文摘要 · Abstract (English)

Question Answering (QA) accounts for a significant portion of LLM usage "in the wild". However, LLMs sometimes produce false or misleading responses, also known as "hallucinations". Therefore, grounding the generated answers in contextually provided information -- i.e., providing evidence for the generated text -- is paramount for LLMs' trustworthiness. Providing this information is the task of context attribution. In this paper, we systematically study LLM-based approaches for this task, namely we investigate (i) zero-shot inference, (ii) LLM ensembling, and (iii) fine-tuning of small LMs on synthetic data generated by larger LLMs. Our key contribution is SynQA: a novel generative strategy for synthesizing context attribution data. Given selected context sentences, an LLM generates QA pairs that are supported by these sentences. This leverages LLMs' natural strengths in text generation while ensuring clear attribution paths in the synthetic training data. We show that the attribution data synthesized via SynQA is highly effective for fine-tuning small LMs for context attribution in different QA tasks and domains. Finally, with a user study, we validate the usefulness of small LMs (fine-tuned on synthetic data from SynQA) in context attribution for QA.

上下文归因数据合成大模型蒸馏可信问答

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。