让大模型多答几次,看答案分歧就能找出它不确定的原因。
Can Multiple Responses from an LLM Reveal the Sources of Its Uncertainty?
- 用多个输出间的不一致模式分析不确定来源。
- 在三个数据集上验证,能识别知识缺失或问题模糊。
- 适合想提升大模型可靠性与可解释性的研究者。
大语言模型在多个领域取得突破,但仍可能生成不可靠或误导性输出,给实际应用带来挑战。尽管已有大量研究关注量化模型不确定性,但针对不确定性的根源诊断工作仍较少。本研究发现,当大模型不确定时,其多个生成回答之间的不一致模式中蕴含着关于不确定原因的丰富线索。为此,我们收集目标大模型的多个响应,并利用一个辅助大模型分析这些响应的分歧模式。该辅助模型被要求推理不确定性的可能来源,例如输入问题的歧义、相关知识缺失,或两者兼有。在存在知识缺口的情况下,辅助模型还能识别出导致不确定的具体缺失事实或概念。我们在 AmbigQA、OpenBookQA 和 MMLU-Pro 上验证了该框架的通用性,结果表明其能有效诊断不同来源的不确定性。这一诊断能力为后续人工干预以提升大模型性能和可靠性提供了可能。
原文摘要 · Abstract (English)
Large language models (LLMs) have delivered significant breakthroughs across diverse domains but can still produce unreliable or misleading outputs, posing critical challenges for real-world applications. While many recent studies focus on quantifying model uncertainty, relatively little work has been devoted to \textit{diagnosing the source of uncertainty}. In this study, we show that, when an LLM is uncertain, the patterns of disagreement among its multiple generated responses contain rich clues about the underlying cause of uncertainty. To illustrate this point, we collect multiple responses from a target LLM and employ an auxiliary LLM to analyze their patterns of disagreement. The auxiliary model is tasked to reason about the likely source of uncertainty, such as whether it stems from ambiguity in the input question, a lack of relevant knowledge, or both. In cases involving knowledge gaps, the auxiliary model also identifies the specific missing facts or concepts contributing to the uncertainty. In our experiment, we validate our framework on AmbigQA, OpenBookQA, and MMLU-Pro, confirming its generality in diagnosing distinct uncertainty sources. Such diagnosis shows the potential for relevant manual interventions that improve LLM performance and reliability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。