用符号推理解码大脑图像响应，让模型理解视觉概念间的逻辑关系。
Neuro-Symbolic Decoding of Neural Activity
- 结合符号推理与脑区活动模式，解析视觉概念间组合关系。
- 在未见过的问题上准确率显著提升，泛化能力更强。
- 适合认知神经科学与具身智能研究者使用。
我们提出 NEURONA，一种用于功能磁共振成像（fMRI）解码与神经活动中概念定位的神经符号框架。基于基于图像和视频的 fMRI 问答数据集，NEURONA 能够根据 fMRI 反应模式，从视觉刺激中解码出相互作用的概念，并在不同脑区间实现符号推理与组合执行的神经定位。实验表明，将结构先验（如概念间的组合谓词-论元依赖关系）引入解码过程，不仅能显著提升对精确查询的解码准确率，还显著增强模型在测试时对未见查询的泛化能力。本研究凸显了神经符号框架在理解神经活动方面的巨大潜力。
原文摘要 · Abstract (English)
We propose NEURONA, a neuro-symbolic framework for fMRI decoding and concept grounding in neural activity. Leveraging image- and video-based fMRI question-answering datasets, NEURONA learns to decode interacting concepts from visual stimuli based on patterns of fMRI responses, integrating symbolic reasoning and compositional execution with fMRI grounding across brain regions. We demonstrate that incorporating structural priors (e.g., compositional predicate-argument dependencies between concepts) into the decoding process significantly improves both decoding accuracy over precise queries, and notably, generalization to unseen queries at test time. With NEURONA, we highlight neuro-symbolic frameworks as promising tools for understanding neural activity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。