用人类知识增强多模态模型，提升医学图像跨域泛化能力
Human Knowledge Integrated Multi-modal Learning for Single Source Domain Generalization
- 结合医学大模型与人类知识，通过低秩适配填补因果差异
- 在8个糖尿病视网膜病变数据集上平均准确率达69.2%
- 适合医疗领域跨中心数据泛化研究者参考
在糖尿病视网膜病变（DR）分级和静息态fMRI癫痫发作起始区（SOZ）检测等关键任务中，跨域图像分类仍具挑战性。当域间存在未知因果因素差异时，难以实现跨域泛化，且缺乏客观评估手段，因通常无法获取数据采集者的元信息或协议级数据。我们首次提出域一致边界（DCB）理论框架，用于评估域间是否在未知因果因素上存在分歧。在此基础上，提出GenEval方法，一种融合基础模型（如MedGemma-4B）与人类知识的多模态视觉语言模型（VLM）方法，通过低秩适配（LoRA）弥合因果差距，增强单源域泛化（SDG）能力。在8个DR数据集和2个SOZ数据集上，GenEval平均准确率分别达到69.2%和81%，优于最强基线9.4%和1.8%。
原文摘要 · Abstract (English)
Generalizing image classification across domains remains challenging in critical tasks such as fundus image-based diabetic retinopathy (DR) grading and resting-state fMRI seizure onset zone (SOZ) detection. When domains differ in unknown causal factors, achieving cross-domain generalization is difficult, and there is no established methodology to objectively assess such differences without direct metadata or protocol-level information from data collectors, which is typically inaccessible. We first introduce domain conformal bounds (DCB), a theoretical framework to evaluate whether domains diverge in unknown causal factors. Building on this, we propose GenEval, a multimodal Vision Language Models (VLM) approach that combines foundational models (e.g., MedGemma-4B) with human knowledge via Low-Rank Adaptation (LoRA) to bridge causal gaps and enhance single-source domain generalization (SDG). Across eight DR and two SOZ datasets, GenEval achieves superior SDG performance, with average accuracy of 69.2% (DR) and 81% (SOZ), outperforming the strongest baselines by 9.4% and 1.8%, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。