arXiv:2606.16234cs.CVcs.AI2026-06中稿 · MICCAI 2026

用OCT结构信息指导,从眼底照片生成血管造影图。

Propagating Structural Guidance: Synthesizing Fluorescein Angiography from Fundus Images and Sparse OCT Scans

论文配图:Propagating Structural Guidance: Synthesizing Fluorescein Angiography from Fundus Images and Sparse OCT Scans
图 1 · 摘自论文原文
  • 融合OCT深度结构与眼底照片纹理,提升血管功能信息重建。
  • 在3676例数据上实现更逼真的造影图合成,诊断准确率更高。
  • 适合眼科医生做无创辅助诊断,尤其缺乏造影设备时使用。

眼底荧光血管造影(FFA)对评估视网膜血管异常至关重要,但其获取具有侵入性且不总可行。相比之下,彩色眼底照相(CFP)无创且普及,促使研究者探索从CFP生成FFA。然而,现有方法仅依赖CFP表面纹理,难以重建功能性的血管信息和细微病理变化。为此,本文提出一种新框架,利用光学相干断层扫描(OCT)提供结构引导,从CFP合成FFA。构建了首个三模态对齐的视网膜影像数据集,包含3,676例患者的眼底照片、FFA和OCT配对数据。为弥合OCT与眼底图像间的空间差异,提出空间对齐跨模态融合(SACMF)模块,将深度分辨的OCT特征投影至眼底平面,并通过自适应层归一化注入到CFP编码器中。此外,引入词元级跨模态对齐(TCMA),在对应空间位置上进行词元级对比学习,显式对齐CFP与FFA表示。实验表明,本方法在合成性能上优于现有最先进方法。大量实验还显示,本方法生成的FFA图像在下游疾病诊断任务中表现更优,凸显其作为无创辅助决策工具的临床潜力。代码已公开于https://github.com/while-plus/OCT-guide-FFA-Syn。

原文摘要 · Abstract (English)

Fundus fluorescein angiography (FFA) is critical for assessing retinal vascular abnormalities, but its acquisition is invasive and not always feasible. In contrast, color fundus photography (CFP) is non-invasive and widely accessible, which has motivated studies on CFP-to-FFA synthesis. However, prior works rely solely on CFP surface texture, fundamentally limiting the ability to reconstruct functional vascular information and subtle pathological changes. To address this, we propose a novel framework that synthesizes FFA from CFP with structural guidance provided by optical coherence tomography (OCT). We construct a multi-modal retinal imaging dataset with paired CFP, FFA, and OCT from 3,676 patient eyes--the first tri-modally aligned dataset in retinal imaging. To bridge the spatial gap between OCT and fundus modalities, we propose a Spatially Aligned Cross-Modal Fusion (SACMF) module that projects depth-resolved OCT features onto the fundus plane and injects them into the CFP encoder via adaptive layer normalization. Beyond feature fusion, we further introduce Token-wise Cross-Modality Alignment (TCMA), a token-level contrastive learning strategy that explicitly aligns CFP and FFA representations at corresponding spatial positions. Our method achieves superior synthesis performance compared to state-of-the-art methods. Moreover, extensive experiments demonstrate that the FFA images synthesized by our approach bring greater improvements in downstream disease diagnosis performance than existing methods, highlighting the clinical potential of our approach as a non-invasive decision-support tool in routine workflows. The code is available at https://github.com/while-plus/OCT-guide-FFA-Syn.

医学影像跨模态生成OCT眼底成像

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。