探索脑影像与图像在神经网络隐空间中的共性，尝试连接人工与自然心智。
Mapping fMRI Signal and Image Stimuli in an Artificial Neural Network Latent Space: Bringing Artificial and Natural Minds Together
- 对比训练于脑影像和图像的自编码器与视觉变换器的隐空间
- 发现两类隐空间存在差异，但结果尚不明确
- 为理解大脑表征与提升模型可解释性提供新思路
本研究旨在探讨视觉刺激与fMRI数据的隐空间表示是否共享共同信息。从fMRI数据中解码和重建视觉刺激仍是人工智能与神经科学领域的挑战，对理解神经表征及提升人工神经网络（ANN）的可解释性具有重要意义。在此初步研究中，我们通过分析一个在fMRI数据上训练的自编码器（AE）与一个在图像数据上训练的视觉变换器（ViT）的隐空间相似性，检验该重建的可行性。采用表示相似性分析（RSA），发现两个域的隐空间存在差异。然而，这些初步结果尚不充分，需进一步深入研究以厘清两者关系。
原文摘要 · Abstract (English)
The goal of this study is to investigate whether latent space representations of visual stimuli and fMRI data share common information. Decoding and reconstructing stimuli from fMRI data remains a challenge in AI and neuroscience, with significant implications for understanding neural representations and improving the interpretability of Artificial Neural Networks (ANNs). In this preliminary study, we investigate the feasibility of such reconstruction by examining the similarity between the latent spaces of one autoencoder (AE) and one vision transformer (ViT) trained on fMRI and image data, respectively. Using representational similarity analysis (RSA), we found that the latent spaces of the two domains appear different. However, these initial findings are inconclusive, and further research is needed to explore this relationship more thoroughly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。