arXiv:2409.19407q-bio.NCcs.AI2024-09NeurIPS被引 87

用新方法建模脑活动,预测能力超现有模型。

Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Masking

  • 采用联合嵌入预测架构,结合梯度定位与时空掩码技术。
  • 在人口统计、疾病诊断等任务上达到当前最佳性能。
  • 适合脑科学与AI交叉研究者,推动神经机制理解。

我们提出Brain-JEPA,一种基于联合嵌入预测架构(JEPA)的脑动态基础模型。该模型在微调后,在人口统计预测、疾病诊断/预后及特质预测任务中达到最先进水平。此外,其在零样本评估(如线性探测)中表现优异,跨不同族裔群体的泛化能力显著优于先前的大规模脑活动模型。Brain-JEPA引入两项创新:脑梯度定位(Brain Gradient Positioning),为脑功能分区建立功能坐标系,增强兴趣区域(ROIs)的位置编码;时空掩码(Spatiotemporal Masking),针对fMRI数据的时间序列异质性设计,有效处理不同时段和空间片段的遮蔽问题。这些方法提升了模型性能,并深化了对认知相关神经回路的理解。总体而言,Brain-JEPA正在为构建脑功能坐标系和在人工智能-神经科学界面实现脑活动掩码提供新范式,通过下游适应开启脑活动分析的新路径。

原文摘要 · Abstract (English)

We introduce Brain-JEPA, a brain dynamics foundation model with the Joint-Embedding Predictive Architecture (JEPA). This pioneering model achieves state-of-the-art performance in demographic prediction, disease diagnosis/prognosis, and trait prediction through fine-tuning. Furthermore, it excels in off-the-shelf evaluations (e.g., linear probing) and demonstrates superior generalizability across different ethnic groups, surpassing the previous large model for brain activity significantly. Brain-JEPA incorporates two innovative techniques: Brain Gradient Positioning and Spatiotemporal Masking. Brain Gradient Positioning introduces a functional coordinate system for brain functional parcellation, enhancing the positional encoding of different Regions of Interest (ROIs). Spatiotemporal Masking, tailored to the unique characteristics of fMRI data, addresses the challenge of heterogeneous time-series patches. These methodologies enhance model performance and advance our understanding of the neural circuits underlying cognition. Overall, Brain-JEPA is paving the way to address pivotal questions of building brain functional coordinate system and masking brain activity at the AI-neuroscience interface, and setting a potentially new paradigm in brain activity analysis through downstream adaptation.

脑科学基础模型fMRI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。