无需训练,用多层级匹配检测医学图像异常
HiMatch-AD: DINOv3-driven Hierarchical Matching for Training-free Medical Anomaly Detection

- 双分支匹配结合全局与局部特征找正常参考
- 跨多阶段生成异常图,精度超越现有方法
- 自适应加权异常响应,适合多模态医疗场景
异常检测在医学图像分析中至关重要,病灶通常表现为正常解剖结构的罕见偏离。尽管基于训练的方法表现优异,但需针对任务优化且依赖大量正常数据,限制了其跨模态和机构的可扩展性。无训练方法通过利用预训练视觉表征提供更高灵活性,但现有方法多依赖简单的最近邻检索和粗暴聚合策略,难以捕捉层次语义并忽略多重异常响应的可靠性。本文提出HiMatch-AD,一种基于DINOv3的无训练医学异常检测框架。首先通过双分支匹配联合考虑全局CLS-token相似性与块级表示,检索语义相关的正常参考;随后在多个Transformer阶段比较聚类后的正常特征与查询表示,生成分层异常图;最后引入统一的不确定性融合机制,根据可靠性自适应加权异常图。整个流程无需任何任务特定训练。在BMAD基准测试中,涵盖脑MRI、肝CT和视网膜OCT数据集,实验表明该方法持续优于训练型及DINO基线方法,验证了多层级匹配与不确定性感知融合的有效性。
原文摘要 · Abstract (English)
Anomaly detection is essential for medical image analysis, where pathological regions often appear as rare deviations from normal anatomical structures. While training-based methods have achieved promising performance, they require task-specific optimization and extensive normal data, which limits scalability across modalities and institutions. Training-free approaches offer greater flexibility by leveraging pretrained visual representations, yet existing methods typically rely on simple nearest-neighbor retrieval and naive aggregation strategies, which may fail to capture hierarchical semantics and ignore the reliability of multiple anomaly responses. In this work, we propose HiMatch-AD, a DINOv3-driven hierarchical matching framework for training-free medical anomaly detection. Our method first retrieves semantically relevant normal references via dual-branch matching that jointly considers global CLS-token similarity and patch-level representations. Hierarchical anomaly maps are then generated across multiple transformer stages by comparing clustered normal features with query representations. To robustly aggregate anomaly responses, we introduce a unified uncertainty-based fusion mechanism that adaptively weights maps according to their reliability. The entire framework operates without any task-specific training. Extensive experiments on the BMAD benchmark, including brain MRI, liver CT, and retinal OCT datasets, demonstrate that HiMatch-AD consistently outperforms both training-based and DINO-based state-of-the-art methods, which highlights the effectiveness of multi-level matching and uncertainty-aware fusion for scalable medical anomaly detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。