arXiv:2603.18846cs.CVcs.LG2026-03被引 1

提出可解释的视网膜图像基础模型,性能媲美大16倍参数的模型。

Towards Interpretable Foundation Models for Retinal Fundus Images

  • 用小感受野的BagNet结构生成决策证据图,实现模型可解释性
  • 在80万张眼底照片上预训练,下游任务表现接近16倍参数的RETFound
  • 通过2D投影可视化表征空间,发现临床有意义聚类与潜在伪相关

基础模型通过自监督学习从大量无标签数据中提取可迁移表征,但在医疗影像等高风险领域,许多模型因架构限制导致可解释性差。本文提出DualIFM,采用BagNet骨干网络,其小感受野能生成忠实反映决策过程的类别证据图,实现设计层面的可解释性。同时,在预训练中引入2D投影层,可直接可视化表征空间,揭示包含有意义临床簇及潜在伪相关性的数据级结构。模型在超过80万张不同来源的彩色眼底照片上训练,学习通用表征以支持多种下游任务。其性能可与参数量多16倍的RETFound相当,且在分布外数据上仍提供可解释预测。结果表明,大规模自监督预训练结合内在可解释性,可获得鲁棒的视网膜成像表征。代码与预训练模型见github.com/berenslab/interpretable_FM。

原文摘要 · Abstract (English)

Foundation models are used to extract transferable representations from large amounts of unlabeled data, typically via self-supervised learning (SSL). However, many of these models rely on architectures that offer limited interpretability, a critical issue in high-stakes domains such as medical imaging. We propose DualIFM, a foundation model that is interpretable-by-design via a BagNet backbone whose small receptive fields generate class evidence maps that are faithful to the model's decision-making process. Additionally, DualIFM incorporates a $2D$ projection layer during pretraining that enables direct visualization of the representation space, providing a dataset-level view of the learned structure including meaningful clinical clusters as well as potential spurious correlations. We trained DualIFM on over 800,000 color fundus photographs from various sources to learn generalizable representations for different downstream tasks. Our model achieves performance comparable to RETFound, which has $16\times$ more parameters, while providing interpretable predictions on out-of-distribution data. These results suggest that large-scale SSL pretraining paired with inherent interpretability can lead to robust representations for retinal imaging. Code and pretrained models are available at github.com/berenslab/interpretable_FM.

可解释模型眼底图像基础模型自监督学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。