用病理描述增强视觉原型,提升多中心糖尿病视网膜病变诊断准确率
Pathology-Aware Prototype Evolution via LLM-Driven Semantic Disambiguation for Multicenter Diabetic Retinopathy Diagnosis
- 构建分层语义提示门控机制,动态选择上下文提示
- 在8个公开数据集上超越现有方法,尤其改善模糊病例判读
- 适合医疗影像分析、多中心疾病诊断研究者参考
糖尿病视网膜病变(DR)分级对早期临床干预和视力保护至关重要。现有方法多聚焦于视觉病灶特征提取,但普遍忽略跨中心一致的病理模式,且仅依赖视觉信息,难以区分细微病理差异。为此,本文提出一种基于大模型语义消歧的病理感知原型演化框架(HAPM)。首先构建基于方差谱的锚点原型库,保留跨域不变的病理特征;其次设计分层差异提示门控机制,从视觉语言模型与大语言模型中动态选择判别性语义提示,缓解相邻分级间的语义混淆;最后采用两阶段原型调制策略,通过病理语义注入器(PSI)与判别原型增强器(DPE)逐步融合临床知识。在八个公开数据集上的实验证明,该方法实现病理引导的原型演化,性能优于当前最优方法。代码已开源。
原文摘要 · Abstract (English)
Diabetic retinopathy (DR) grading plays a critical role in early clinical intervention and vision preservation. Recent explorations predominantly focus on visual lesion feature extraction through data processing and domain decoupling strategies. However, they generally overlook domain-invariant pathological patterns and underutilize the rich contextual knowledge of foundation models, relying solely on visual information, which is insufficient for distinguishing subtle pathological variations. Therefore, we propose integrating fine-grained pathological descriptions to complement prototypes with additional context, thereby resolving ambiguities in borderline cases. Specifically, we propose a Hierarchical Anchor Prototype Modulation (HAPM) framework to facilitate DR grading. First, we introduce a variance spectrum-driven anchor prototype library that preserves domain-invariant pathological patterns. We further employ a hierarchical differential prompt gating mechanism, dynamically selecting discriminative semantic prompts from both LVLM and LLM sources to address semantic confusion between adjacent DR grades. Finally, we utilize a two-stage prototype modulation strategy that progressively integrates clinical knowledge into visual prototypes through a Pathological Semantic Injector (PSI) and a Discriminative Prototype Enhancer (DPE). Extensive experiments across eight public datasets demonstrate that our approach achieves pathology-guided prototype evolution while outperforming state-of-the-art methods. The code is available at https://github.com/zhcz328/HAPM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。