arXiv:2604.08537cs.LGq-bio.NC2026-04中稿 · CVPR被引 1

无需微调,用少量数据即可跨被试解码脑信号中的视觉信息。

Meta-learning In-Context Enables Training-Free Cross Subject Brain Decoding

论文配图:Meta-learning In-Context Enables Training-Free Cross Subject Brain Decoding
图 1 · 摘自论文原文
  • 通过上下文学习快速推断新被试的神经编码模式。
  • 在多个脑区实现无训练跨被试、跨扫描仪解码,准确率高。
  • 适合构建通用非侵入式脑解码基础模型的研究者使用。

从脑信号中进行视觉解码是计算机视觉与神经科学交叉领域的关键挑战,需建立神经表征与视觉计算模型之间的桥梁。当前目标是实现可泛化的跨被试模型,但个体间神经表征差异大,传统方法需为每个被试单独训练或微调。本文提出一种元优化的fMRI语义视觉解码方法,无需任何微调即可泛化至新被试。仅通过少量新被试的图像-脑激活样例作为上下文,模型即可快速推断其独特的神经编码模式,实现稳健高效的视觉解码。方法采用分层推理:首先在多个脑区上,基于多刺激-响应上下文估计每体素的视觉响应编码参数;其次,结合编码参数与响应值,对多体素进行聚合功能反演。实验表明,该方法在多种视觉骨干网络下均表现出强跨被试和跨扫描仪泛化能力,且无需解剖对齐或刺激重叠。本工作为构建非侵入式脑解码通用基础模型迈出关键一步。

原文摘要 · Abstract (English)

Visual decoding from brain signals is a key challenge at the intersection of computer vision and neuroscience, requiring methods that bridge neural representations and computational models of vision. A field-wide goal is to achieve generalizable, cross-subject models. A major obstacle towards this goal is the substantial variability in neural representations across individuals, which has so far required training bespoke models or fine-tuning separately for each subject. To address this challenge, we introduce a meta-optimized approach for semantic visual decoding from fMRI that generalizes to novel subjects without any fine-tuning. By simply conditioning on a small set of image-brain activation examples from the new individual, our model rapidly infers their unique neural encoding patterns to facilitate robust and efficient visual decoding. Our approach is explicitly optimized for in-context learning of the new subject's encoding model and performs decoding by hierarchical inference, inverting the encoder. First, for multiple brain regions, we estimate the per-voxel visual response encoder parameters by constructing a context over multiple stimuli and responses. Second, we construct a context consisting of encoder parameters and response values over multiple voxels to perform aggregated functional inversion. We demonstrate strong cross-subject and cross-scanner generalization across diverse visual backbones without retraining or fine-tuning. Moreover, our approach requires neither anatomical alignment nor stimulus overlap. This work is a critical step towards a generalizable foundation model for non-invasive brain decoding.

脑解码元学习跨被试上下文学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。