用医学报告指导专家模型，提升眼底OCT/OCTA异常检测精度
ReMoE: Report-Guided Mixture-of-Experts for Multimodal OCT/OCTA Anomaly Detection
- 利用正常报告语义构建图文先验，指导多模态特征学习
- 在私有数据集和OCTA500-3MM上达到最新最好性能
- 适合需要结合临床报告的医学影像异常检测场景
多模态医学异常检测旨在识别与正常模式偏离的样本，由于异常病例稀少，仅基于正常数据建模正常性是可行的。在视网膜光学相干断层扫描（OCT）和OCT血管成像（OCTA）异常检测中，现有无监督方法依赖视觉特征分布、重建残差或编码器-解码器差异，使异常评分仅反映外观层面偏差，而多模态正常性还包含正常医学报告中的语义结构。为此，我们提出报告引导的专家混合模型（ReMoE），将正常报告语义提炼为图像到文本的先验学生模型，构建模态感知先验，并通过报告引导的模态调制（RMM）机制，利用专家混合路由调节特征表示。在包含配对正常报告的私有OCT/OCTA数据集及使用固定正常报告的公开OCTA500-3MM设置上进行实验，均取得当前最佳性能。
原文摘要 · Abstract (English)
Multimodal medical anomaly detection identifies samples deviating from normal patterns, where scarce abnormal cases make normality modeling from normal data practical. In retinal Optical Coherence Tomography (OCT) and OCT Angiography (OCTA) anomaly detection, existing unsupervised methods rely on visual feature distributions, reconstruction residuals, or encoder-decoder discrepancies, making anomaly scores depend on appearance-level deviations, while multimodal normality also contains semantic organization described in normal medical reports. To this end, we propose Report-Guided Mixture-of-Experts (ReMoE), which distills normal report semantics into an image-to-text prior student, builds modality-aware priors, and uses Report-Guided Modality Modulation (RMM) to modulate features through mixture-of-experts routing. Experiments on a private OCT/OCTA dataset with paired normal reports and a public OCTA500-3MM setting using a fixed normal report demonstrate state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。