通过量化输入模糊性提升大模型医疗问答安全性
Mind the Ambiguity: Aleatoric Uncertainty Quantification in LLMs for Safe Medical Question Answering
- 从内部激活模式中识别出输入模糊性对应的随机不确定性
- 提出无需微调的轻量级探针模块,平均提升准确率9.48%
- 适合医疗AI安全评估与可解释性研究者使用
大型语言模型在医疗问答中的应用因用户查询模糊而面临严重安全隐患,显著降低高风险医疗场景下的回答准确性。本文将输入模糊性与不可消除的随机不确定性(Aleatoric Uncertainty, AU)建立关联,并构建首个针对医疗QA中输入模糊性的基准数据集CV-MedBench。通过该数据集,我们从表征工程角度分析发现,AU在线性编码于大模型内部激活模式中。基于此发现,提出一种“先澄清再回答”的新框架,引入轻量级的AU-Probe模块,直接从隐藏状态检测输入模糊性。该方法无需模型微调或多次前向传播,可高效触发用户澄清请求,显著提升安全性。在四个开源大模型上进行的大量实验表明,该框架相较基线平均提升准确率9.48%。代码与数据集均已公开。
原文摘要 · Abstract (English)
The deployment of Large Language Models in Medical Question Answering is severely hampered by ambiguous user queries, a significant safety risk that demonstrably reduces answer accuracy in high-stakes healthcare settings. In this paper, we formalize this challenge by linking input ambiguity to aleatoric uncertainty (AU), which is the irreducible uncertainty arising from underspecified input. To facilitate research in this direction, we construct CV-MedBench, the first benchmark designed for studying input ambiguity in Medical QA. Using this benchmark, we analyze AU from a representation engineering perspective, revealing that AU is linearly encoded in LLM's internal activation patterns. Leveraging this insight, we introduce a novel AU-guided "Clarify-Before-Answer" framework, which incorporates AU-Probe - a lightweight module that detects input ambiguity directly from hidden states. Unlike existing uncertainty estimation methods, AU-Probe requires neither LLM fine-tuning nor multiple forward passes, enabling an efficient mechanism to proactively request user clarification and significantly enhance safety. Extensive experiments across four open LLMs demonstrate the effectiveness of our QA framework, with an average accuracy improvement of 9.48% over baselines. Our framework provides an efficient and robust solution for safe Medical QA, strengthening the reliability of health-related applications. The code is available at https://github.com/yaokunliu/AU-Med.git, and the CV-MedBench dataset is released on Hugging Face at https://huggingface.co/datasets/yaokunl/CV-MedBench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。