无需训练即可检测医疗摘要中的来源和幻觉,提升生成可信度。
From Attribution to Abstention: Training-Free Attention-Based Auditing for Clinical Summarization
- 利用Transformer解码器注意力权重,一次计算获得来源定位与支持度评分。
- 在放射科和对话摘要任务中,来源定位准确率超92%,幻觉检测AUROC达0.77。
- 适用于医疗AI系统,帮助医生识别不可信内容并选择性拒答。
将多模态大语言模型(MLLM)用于临床摘要需兼顾生成流畅性与透明性,明确每句话的来源,并能标识缺乏证据支持的陈述。我们提出ClinTrace,一种无需训练的框架,从每个基于Transformer的MLLM生成时产生的解码器注意力权重中提取两个临床有用信号:(i) 精细粒度的来源归因,将输出句子关联到支持文本片段或图像;(ii) 每句的扎根度评分,识别支持不足的主张作为潜在幻觉候选。两者均来自同一注意力张量的一次计算,无需重训练、无附加模型或推理成本。我们在两个临床摘要任务上评估:医生-患者对话摘要(CliConSummation)和放射科报告摘要(MIMIC-CXR),使用通用模型Qwen3-8B和医学微调模型HuatuoGPT-Vision-7B。在来源归因上,ClinTrace在放射科达到超过92%的文本F1,对话任务达88%,显著优于基于嵌入和自归因的基线。在幻觉检测上,扎根度评分在医学微调模型下取得0.77 AUROC,媲美基于嵌入的置信度且零额外成本,还可实现拒答机制,通过保留最不扎根的20%输出供医生审查,使忠实度从61.7%提升至72.6%。值得注意的是,医学微调显著提升了基于注意力的幻觉检测可靠性,表明领域适配使注意力模式更具语义结构,更利于自审计。
原文摘要 · Abstract (English)
Deploying multimodal large language models (MLLMs) for clinical summarization demands not only fluent generation but also transparency about where each statement originates-and a mechanism to flag when statements lack evidential support. We present ClinTrace, a training-free framework that extracts two clinically useful signals from the decoder attention weights that every transformer-based MLLM already produces during generation: (i) fine-grained source attributions linking each output sentence to supporting text spans or images, and (ii) per-sentence groundedness scores that identify poorly supported claims as candidate hallucinations. Both signals are derived from the same attention tensors in a single pass, requiring no retraining, no auxiliary models, and no additional inference cost. We evaluate on two clinical summarization tasks: doctor-patient dialogue summarization (CliConSummation) and radiology report summarization (MIMIC-CXR) using a general-purpose MLLM (Qwen3-8B) and a medical-finetuned model (HuatuoGPT-Vision-7B). For source attribution, ClinTrace achieves over 92% text F1 on radiology and 88% on dialogue summarization, substantially outperforming embedding-based and self-attribution baselines. For hallucination detection, groundedness scores achieve 0.77 AUROC with the medical-finetuned model: competitive with embedding-based confidence at zero additional cost, and enable an abstention mechanism that improves faithfulness from 61.7% to 72.6% by withholding the least: grounded 20% of output for clinician review. Notably, medical finetuning substantially improves the reliability of attention-based hallucination detection, suggesting that domain adaptation produces more semantically structured attention patterns amenable to self-auditing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。