通过行为审计揭示医疗大模型如何使用病历证据，发现准确诊断下仍存在证据误用。
Auditing Evidence Use in Medical LLM Diagnosis
- 将患者信息拆解为证据单元，控制性测试诊断得分变化
- 多数证据交互在临床逻辑上合理，非模型失败
- 适合关注医疗AI可解释性与安全性的研究者和临床评估人员
医疗大模型常以诊断正确性评估，但准确性无法反映模型是否合理使用病历证据。本文提出一种行为审计方法：将病例信息分解为证据单元，在受控证据子集下评分候选诊断，并挖掘诊断置信度差异中的低阶交互。由于医学证据具有诊断相关性，审计将交互发现与错误归因分离——强或负向交互可能反映合理鉴别诊断，可疑交互则需进一步稳健性检验与临床审查。我们在DDXPlus、CupCase和MedCase数据集上评估了五种开源大模型。结果显示，多数交互强度由忠实支持和鉴别冲突/抵消主导，表明多数交互具有临床合理性而非模型缺陷。在针对DDXPlus的130项盲审增强样本（5名评审员）中，无效或捷径式案例集中于否定或缺失发现及局部临床证据。结果表明，诊断准确率可能掩盖证据使用缺陷，呼吁开展角色感知的审计以完善医疗大模型评估。
原文摘要 · Abstract (English)
Medical LLMs are often evaluated by whether they select the correct diagnosis, but diagnostic accuracy alone does not show whether the model used the case evidence appropriately. We present a behavioral audit of evidence use in medical diagnosis. For each case, we decompose patient information into evidence units, score candidate diagnoses under controlled evidence subsets, and mine low-order interactions in diagnostic margins. Because medical evidence is diagnosis-relative, the audit separates interaction discovery from failure assignment: large or negative interactions can reflect plausible differential diagnosis, while suspicious interactions require robustness checks and clinical review. We evaluate five open-weight LLMs on DDXPlus, CupCase, and MedCase. Across datasets, faithful support and differential conflict or cancellation account for most interaction strength, showing that many evidence interactions are clinically plausible rather than failures. In a DDXPlus-focused blinded five-reviewer 130-item enriched review sample, invalid or shortcut-like cases concentrate in negated or absent findings and clinically local evidence. These results show that accuracy can hide candidate evidence-use failures and motivate role-aware audits for medical LLM evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。