arXiv:2508.10298cs.LGcs.CV2025-08NeurIPS被引 11

用概率模型模拟视觉到脑影像的映射,提升生成质量与生物可解释性。

SynBrain: Enhancing Visual-to-fMRI Synthesis via Probabilistic Representation Learning

  • 基于概率分布建模神经响应,结合视觉语义约束保持功能一致性。
  • 在跨被试、少样本条件下仍能生成高质量fMRI信号,优于现有方法。
  • 适合研究脑科学中的神经编码机制或需低数据量生成的医学影像任务。

解析视觉刺激如何转化为皮层响应是计算神经科学的核心挑战。这一视觉到神经的映射本质上是一对多关系,相同视觉输入在不同实验条件和被试间会引发可重复但变化的血流动力学响应。现有确定性方法难以同时捕捉这种生物变异性与隐含的功能一致性。为此,我们提出SynBrain,一种生成式框架,以概率化且生物可解释的方式模拟从视觉语义到神经响应的转换。SynBrain引入两个关键组件:(i) BrainVAE通过概率学习将神经表征建模为连续分布,同时利用视觉语义约束维持功能一致性;(ii) 语义到神经映射器作为语义传输路径,将视觉语义投影至神经响应流形,实现高保真度fMRI合成。实验表明,SynBrain在个体特异性的视觉到fMRI编码性能上超越当前最优方法。此外,它在少量新被试数据下即可高效适配,并生成有效提升数据稀缺条件下fMRI到图像解码性能的高质量信号。更进一步,SynBrain揭示了跨试验和被试的功能一致性,合成信号展现出由生物神经变异性塑造的可解释模式。代码已开源:https://github.com/MichaelMaiii/SynBrain。

原文摘要 · Abstract (English)

Deciphering how visual stimuli are transformed into cortical responses is a fundamental challenge in computational neuroscience. This visual-to-neural mapping is inherently a one-to-many relationship, as identical visual inputs reliably evoke variable hemodynamic responses across trials, contexts, and subjects. However, existing deterministic methods struggle to simultaneously model this biological variability while capturing the underlying functional consistency that encodes stimulus information. To address these limitations, we propose SynBrain, a generative framework that simulates the transformation from visual semantics to neural responses in a probabilistic and biologically interpretable manner. SynBrain introduces two key components: (i) BrainVAE models neural representations as continuous probability distributions via probabilistic learning while maintaining functional consistency through visual semantic constraints; (ii) A Semantic-to-Neural Mapper acts as a semantic transmission pathway, projecting visual semantics into the neural response manifold to facilitate high-fidelity fMRI synthesis. Experimental results demonstrate that SynBrain surpasses state-of-the-art methods in subject-specific visual-to-fMRI encoding performance. Furthermore, SynBrain adapts efficiently to new subjects with few-shot data and synthesizes high-quality fMRI signals that are effective in improving data-limited fMRI-to-image decoding performance. Beyond that, SynBrain reveals functional consistency across trials and subjects, with synthesized signals capturing interpretable patterns shaped by biological neural variability. Our code is available at https://github.com/MichaelMaiii/SynBrain.

脑影像生成概率建模神经编码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。