用博弈论方法精确定位输入模糊处，让大模型知道哪里该澄清。
Localizing Input Uncertainty Quantification for Large Language Models via Shapley Values
- 把输入片段看作博弈玩家，用谢林值计算各自对不确定性的贡献。
- 在两个基准上超越现有方法，准确识别输入中的模糊部分。
- 适合医疗等高风险场景，帮人机协作精准改进输入质量。
随着大语言模型越来越多地应用于高风险决策，可靠地量化不确定性已成为安全与可信的关键需求。然而，当前的不确定性量化方法多聚焦于输出层面，难以区分不确定性是源于模型知识不足还是用户输入模糊。尽管以输入为中心的不确定性量化逐渐兴起，但大多依赖粗粒度输入信息，仅提供标量分数，无法指导用户应澄清输入的哪些部分。为此，我们提出基于谢林值的输入不确定性量化框架（ShaQ），实现对输入片段级不确定性的归因。该方法将输入中的模糊片段视为合作博弈中的参与者，通过加权平均边际熵减少量来定义其贡献。相比现有方法，我们的方法能捕捉片段间的复杂交互，并保证个体归因之和等于总输入诱导的不确定性。我们在AmbigQA和AmbiEnt基准上评估，结果达到领先水平。进一步在MediTOD上验证，证明其可定位临床语句中表述不清的部分，促进人机协作。整体而言，ShaQ提升了不确定性估计能力，并为针对性输入修正提供可操作洞察。
原文摘要 · Abstract (English)
As large language models (LLMs) are increasingly integrated into high-stakes decision-making, the ability to reliably quantify uncertainty has become a critical requirement for safety and trust. However, current uncertainty quantification methods primarily operate at the output level, often failing to distinguish whether uncertainty arises from the model's lack of knowledge or from ambiguity in the user's input. While input-centric uncertainty quantification has recently emerged as a promising direction, it remains relatively underexplored and typically relies on coarse, input-level information. Consequently, users are provided with scalar uncertainty scores that offer little actionable guidance on which parts of the input should be clarified to improve reliability. To address this limitation, we propose Shapley-based input uncertainty Quantification (ShaQ), a framework for span-level attribution of input-induced uncertainty. Our approach models ambiguous spans in the input as players in a cooperative game and quantifies their contributions using Shapley values, defined via the weighted average of marginal reductions in conditional entropy obtained by clarifying each span coalition. Unlike existing input-level approaches, our formulation captures complex interactions among spans and provides a principled decomposition in which individual attributions sum exactly to the total input-induced uncertainty. We evaluate ShaQ on the AmbigQA and AmbiEnt benchmarks, where it achieves state-of-the-art performance in ambiguity detection. We further demonstrate its utility on MediTOD, showing that ShaQ can localize under-specified clinical utterances and facilitate human-AI collaboration in high-stakes settings. Overall, ShaQ improves uncertainty estimation and provides actionable insights for targeted input clarification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。