LLM在临床表格数据上常盲目自信,本文用跨模型归因差异检测其知识盲区。
LLM Doesn't Know What It Doesn't Know: Detecting Epistemic Blind Spots via Cross-Model Attribution Divergence on Clinical Tabular Data
- 通过对比LLM与XGBoost的归因分歧,识别其认知盲点。
- 引入少样本示例和SHAP特征证据,使准确率从49%提升至75.3%。
- 无需训练即可生成患者级可靠性评估,替代无效的置信度输出。
大型语言模型(LLMs)在结构化临床数据上的应用日益广泛,但其对自身知识边界的认识仍不清楚。本文通过跨模型归因分歧分析,研究了在预测任务中减少认知不确定性的问题,对比了Qwen 2.5 7B与XGBoost的表现。结果发现:第一,LLM的口头置信度近乎恒定(0.856–0.937),与实际准确率无关,仅受提示格式影响;第二,当XGBoost准确率达99%时,LLM准确率降至64.8%,但在中等不确定时能匹配其表现(73.8% vs. 73.1%);第三,少量示例与SHAP特征证据为正交增强,使归因分歧得分(ADS)从1.54降至0.38,准确率从49%升至75.3%,无需训练;第四,基于归因分歧构建的跨模型校准器,将期望校准误差从0.254降至0.080,实现无需访问模型内部或重复推理的患者级可靠性估计。这些发现揭示了结构化任务中LLM的冷启动困境,并指明走向真正认知自知的路径。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly applied to structured clinical data, yet whether they can recognize the limits of their own knowledge on such tasks remains unexplored. We study this question through the lens of cross-model attribution divergence with the goal of reducing epistemic uncertainty for structured tasks, comparing Qwen 2.5 7B and XGBoost on a prediction task via attribution divergence analysis. We report four findings. First, LLM verbalized confidence is epistemically vacuous, it outputs a near-constant (0.856-0.937) regardless of whether accuracy is 49% or 75.3%, tracking prompt format rather than prediction quality. Second, the LLM exhibits an inverse difficulty effect: accuracy drops to 64.8% when XGBoost is 99% correct, but matches XGBoost (73.8% vs. 73.1%) when it is moderately uncertain. Third, few-shot examples and SHAP-derived feature evidence are orthogonal, super-additive interventions: they reduce the Attribution Disagreement Score (ADS) from 1.54 to 0.38 and improve accuracy from 49% to 75.3% without training. Fourth, a cross-model calibrator that determined LLM reliability using attribution divergence signals reduces expected calibration error from 0.254 to 0.080, replacing uninformative verbalized confidence with patient-specific reliability estimates, without accessing model internals or requiring repeated inference. We frame these findings as a cold start problem for LLMs on structured data and outline a path toward genuine epistemic self-awareness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。