arXiv:2603.29376cs.CV2026-03被引 1

用专家判断评估多模态伤口表型嵌入,提升罕见病相似病例发现能力。

Assessing Multimodal Chronic Wound Embeddings with Expert Triplet Agreement

  • 通过专家三元组判断构建临床语义感知的嵌入空间
  • 多模态融合使专家一致率达73.5%,优于单模态模型5.6个百分点
  • 适用于罕见皮肤病研究与临床辅助诊断场景

隐性营养不良性大疱性表皮松解症(RDEB)是一种罕见遗传性皮肤病,临床中需借助图像与文本匹配相似病例。现有通用基础模型难以捕捉该异质性、长尾疾病的临床意义特征,且专家一致性评估困难。为此,本文提出基于专家序数比较(三元组判断)的嵌入空间评估方法,该方法采集快速,可编码隐含临床相似性知识。进一步提出TriDerm多模态框架,通过整合伤口图像、边界掩码与专家报告,从小规模队列中学习可解释的伤口表型表示。视觉部分采用伤口级注意力池化与非对比式表示学习适配视觉基础模型;文本部分通过对比查询提示大语言模型,以软序数嵌入(SOE)恢复医学意义表示。实验表明,视觉与文本模态捕获互补的表型信息,两者融合后与专家的一致率达到73.5%,显著优于最佳单模态基础模型超5.6个百分点。相关专家标注工具、模型代码及代表性数据样本已公开。

原文摘要 · Abstract (English)

Recessive dystrophic epidermolysis bullosa (RDEB) is a rare genetic skin disorder for which clinicians greatly benefit from finding similar cases using images and clinical text. However, off-the-shelf foundation models do not reliably capture clinically meaningful features for this heterogeneous, long-tail disease, and structured measurement of agreement with experts is challenging. To address these gaps, we propose evaluating embedding spaces with expert ordinal comparisons (triplet judgments), which are fast to collect and encode implicit clinical similarity knowledge. We further introduce TriDerm, a multimodal framework that learns interpretable wound representations from small cohorts by integrating wound imagery, boundary masks, and expert reports. On the vision side, TriDerm adapts visual foundation models to RDEB using wound-level attention pooling and non-contrastive representation learning. For text, we prompt large language models with comparison queries and recover medically meaningful representations via soft ordinal embeddings (SOE). We show that visual and textual modalities capture complementary aspects of wound phenotype, and that fusing both modalities yields 73.5% agreement with experts, outperforming the best off-the-shelf single-modality foundation model by over 5.6 percentage points. We make the expert annotation tool, model code and representative dataset samples publicly available.

多模态医学影像嵌入学习罕见病

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。