arXiv:2605.28047cs.CL2026-05

给大模型问答的依据打分,识别关键知识来源。

Knowledge Dependency Estimation for Reliable Question Answering

论文配图:Knowledge Dependency Estimation for Reliable Question Answering
图 1 · 摘自论文原文
  • 通过反事实监督学习,评估不同知识单元对答案的影响程度。
  • 在多个基准上优于基线,能准确识别影响答案的关键知识项。
  • 无需额外调用模型,适合用于提前发现高风险问答结果。

可靠的问题回答不仅需要判断答案正确性,还需明确预测所依赖的知识来源。在基于大模型的问答中,这些知识可能来自上下文、检索结果、分解推理或中间推导过程,形成噪声大且冗余的候选知识空间,而非纯净的黄金证据集。本文研究“知识依赖性估计”:估计固定黑箱问答模型对不同候选知识单元的敏感度。挑战在于不进行全量测试时扰动,仍需精细捕捉依赖关系,并建模冗余、可替代性和互补性。我们提出结构化感知排序的依赖估计器 Knot,通过子集级反事实监督学习,利用潜在依赖因子的覆盖度建模子集敏感性,并生成感知排序的单元得分,以识别关键候选。在多项选择和生成式问答基准上,Knot 在子集敏感性预测上超越所有对比基线,且无需额外调用问答模型即可生成更可信的单元排名;用于实际风险筛查时,其依赖得分可有效提前标记易错问答预测。

原文摘要 · Abstract (English)

Reliable question answering requires identifying not only whether an answer is correct, but also which available knowledge the prediction depends on. In realistic LLM-based QA, this knowledge may come from context, retrieval, decomposition, or intermediate reasoning, forming a noisy and redundant candidate space rather than a clean gold evidence set. We study \emph{knowledge dependency estimation}: estimating the sensitivity of a fixed black-box QA model to different candidate knowledge units. The challenge is to obtain fine-grained dependency scores without exhaustive test-time perturbation while modeling redundancy, substitutability, and complementarity. We propose \textbf{Knot}, a structured rank-aware knowledge dependency estimator. Knot learns from subset-level counterfactual supervision, models subset sensitivity through coverage over latent dependency factors, and derives rank-aware unit scores to identify influential candidates. Across multiple-choice and generative QA benchmarks, Knot outperforms all compared baselines in subset-sensitivity prediction and produces more faithful unit rankings than deployable baselines without extra QA-model calls; when used for practical risk screening, its dependency scores help flag error-prone QA predictions early.

问答系统知识依赖模型解释风险检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。