用静息态脑图预测视觉刺激下的脑活动,实现低成本个体化功能建模。
Rest2Visual: Predicting Visually Evoked fMRI from Resting-State Scans
- 基于静息态fMRI和图像嵌入,通过自适应归一化生成刺激特异性激活图
- 预测结果在相似性和表征度量上接近真实数据,支持后续图像重建
- 可生成具有个体差异的神经替代数据,适合大规模脑科学研究
理解自发脑活动与刺激驱动神经反应的关系是认知神经科学中的基本挑战。任务态功能性磁共振成像(fMRI)虽能捕捉局部刺激响应,但采集成本高、耗时长且难以规模化。相比之下,静息态fMRI(rs-fMRI)无需任务、数据丰富,但缺乏直接可解释性。我们提出Rest2Visual,一种条件生成模型,从静息态输入和2D视觉刺激预测视觉诱发电码(ve-fMRI)。该模型采用体积分解器-解码器结构,利用多尺度3D特征结合图像嵌入,通过自适应归一化实现空间精确、刺激特定的激活合成。为支持训练,我们基于自然场景数据集(NSD)构建了大规模三元组数据集,将每个rs-fMRI体积与对应刺激图像及ve-fMRI激活图对齐。定量评估显示,预测激活在标准相似性和表征度量上与真实数据高度一致,并支持下游解码中的图像重建。值得注意的是,预测图保留了受试者特异性结构,证明模型具备生成个体化功能替代数据的能力。结果表明,个体自发神经活动可被转化为与刺激对齐的表示,为可扩展、无任务的功能脑建模开辟新路径。
原文摘要 · Abstract (English)
Understanding how spontaneous brain activity relates to stimulus-driven neural responses is a fundamental challenge in cognitive neuroscience. While task-based functional magnetic resonance imaging (fMRI) captures localized stimulus-evoked brain activation, its acquisition is costly, time-consuming, and difficult to scale across populations. In contrast, resting-state fMRI (rs-fMRI) is task-free and abundant, but lacks direct interpretability. We introduce Rest2Visual, a conditional generative model that predicts visually evoked fMRI (ve-fMRI) from resting-state input and 2D visual stimuli. It follows a volumetric encoder--decoder design, where multiscale 3D features from rs-fMRI are modulated by image embeddings via adaptive normalization, enabling spatially accurate, stimulus-specific activation synthesis. To enable model training, we construct a large-scale triplet dataset from the Natural Scenes Dataset (NSD), aligning each rs-fMRI volume with stimulus images and their corresponding ve-fMRI activation maps. Quantitative evaluation shows that the predicted activations closely match ground truth across standard similarity and representational metrics, and support successful image reconstruction in downstream decoding. Notably, the predicted maps preserve subject-specific structure, demonstrating the model's capacity to generate individualized functional surrogates. Our results provide compelling evidence that individualized spontaneous neural activity can be transformed into stimulus-aligned representations, opening new avenues for scalable, task-free functional brain modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。