让脑影像与病历文本对齐更透明公平,避免算法偏见。
NeuroLIP: Interpretable and Fair Cross-Modal Alignment of fMRI and Phenotypic Text
- 用文本词元控制注意力,精准定位疾病相关脑区。
- 在ABIDE和ADHD-200数据集上公平性提升,诊断准确率最优。
- 适合关注神经影像AI可解释性与公平性的研究者。
将功能磁共振成像(fMRI)连接数据与表型文本描述(如疾病标签、人口统计信息)融合,有助于深入理解神经疾病。但现有跨模态对齐方法常缺乏可解释性,且可能将敏感属性与诊断特征混编,引入偏见。本文提出NeuroLIP,一种新型跨模态对比学习框架。通过引入文本词元条件注意力(TTCA)和基于局部词元的跨模态对齐(CALT),在脑区级嵌入中实现疾病相关词元的显式建模。该设计生成词元级别注意力图,揭示脑区与疾病的关联,增强可解释性。为缓解偏见,提出敏感属性解耦损失,最大化疾病词元与敏感属性词元间的注意力距离,降低下游预测中的非预期相关性。此外,采用负梯度技术,反转敏感属性上的CALT损失符号,进一步抑制其对齐。在ABIDE和ADHD-200数据集上的实验表明,NeuroLIP在公平性指标上表现更优,同时保持最佳标准性能。注意力图的定性可视化显示了与诊断特征一致的神经解剖模式,得到神经科学文献支持。本工作推动了透明且公平的神经影像人工智能发展。
原文摘要 · Abstract (English)
Integrating functional magnetic resonance imaging (fMRI) connectivity data with phenotypic textual descriptors (e.g., disease label, demographic data) holds significant potential to advance our understanding of neurological conditions. However, existing cross-modal alignment methods often lack interpretability and risk introducing biases by encoding sensitive attributes together with diagnostic-related features. In this work, we propose NeuroLIP, a novel cross-modal contrastive learning framework. We introduce text token-conditioned attention (TTCA) and cross-modal alignment via localized tokens (CALT) to the brain region-level embeddings with each disease-related phenotypic token. It improves interpretability via token-level attention maps, revealing brain region-disease associations. To mitigate bias, we propose a loss for sensitive attribute disentanglement that maximizes the attention distance between disease tokens and sensitive attribute tokens, reducing unintended correlations in downstream predictions. Additionally, we incorporate a negative gradient technique that reverses the sign of CALT loss on sensitive attributes, further discouraging the alignment of these features. Experiments on neuroimaging datasets (ABIDE and ADHD-200) demonstrate NeuroLIP's superiority in terms of fairness metrics while maintaining the overall best standard metric performance. Qualitative visualization of attention maps highlights neuroanatomical patterns aligned with diagnostic characteristics, validated by the neuroscientific literature. Our work advances the development of transparent and equitable neuroimaging AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。