提出新方法评估超声心动图模型的注意力图可靠性,发现传统重叠指标不可靠。
Ablation-Corrected Evaluation of Attribution Maps in Echocardiographic Ejection-Fraction Models
- 用对称消融法生成真实随机基准,替代传统面积比估算
- 两模型重叠度差异仅18%,但实际依赖性差异达3倍以上
- 建议报告消融基准、可靠性与预测响应,而非单一重叠值
本研究通过模型预测对左心室标注区域的依赖性,评估超声心动图射血分数模型的注意力图。使用对称消融法构建真实随机基准,取代传统的面积比估算。两个同任务训练的模型在该基准下重叠度分别为3.55和4.20倍实测随机水平,看似相差18%;但实际消融实验显示,一个模型对心室替换响应强度是对照区的六倍,另一模型仅为两倍(差值三倍)。在62%情况下,第二个模型对心室或注意力区域的改变均无预测变化。独立干预批次的可靠性为0.90至0.98。以消融结果作为标签评估时,重叠指标的AUC仅为0.52和0.67。消融策略影响结果:仅消融两帧时心室无法区分于对照区,而全帧消融则可区分。儿科队列复现所有现象。本文建议报告实测随机基准、对称消融协议、可靠性估计及消融-预测评分。
原文摘要 · Abstract (English)
Attribution maps for echocardiographic ejection-fraction models are evaluated by their overlap with an expert left-ventricular annotation, compared against a chance level that is computed from an area ratio rather than measured. We measure it. Two architectures trained on the same task attain overlap at 3.55 and 4.20 times measured chance, an eighteen percent difference a reader would take as the size of the gap between them. It is not. Replacing the annotated ventricle with a composition-matched surrogate changes one model's prediction six times more than replacing an equal-area control region and the other's twice, a factor of three; on sixty-two percent of cases for the second, neither the ventricle nor the attributed region moves the prediction at all. The reliability of this measurement, from two independent intervention batches, is 0.90 to 0.98. Overlap does not track the difference: scored against an ablation-derived label of which cases the model depends on the ventricle for, it reaches an area under the curve of 0.52 and 0.67. The design of the ablation also determines its answer, since ablating only the two annotated frames leaves the ventricle indistinguishable from a control region while ablating across all frames does not. A pediatric cohort reproduces all of these. We give the measured chance level, the symmetric-ablation protocol, the reliability estimate, and the ablation-prediction score as things to report alongside overlap.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。