arXiv:2604.15374q-bio.NCcs.AI2026-04

用预训练模型解码想象中的图像,提升脑信号到视觉内容的还原准确率。

Seeing the imagined: a latent functional alignment in visual imagery decoding from fMRI data

论文配图:Seeing the imagined: a latent functional alignment in visual imagery decoding from fMRI data
图 1 · 摘自论文原文
  • 将想象时的脑活动映射到生成模型的条件空间,保持其余部分冻结。
  • 在4名受试者中实现高于随机水平的语义重建,尤其在高级皮层区域有效。
  • 适合脑机接口、认知神经科学领域研究者参考。

最近的视觉脑解码进展得益于大规模数据集(如自然场景数据集NSD)和强大的基于扩散的生成模型。尽管当前流程主要针对感知优化,其在心理想象任务下的表现仍不清晰。本文研究如何将最先进的感知解码器(DynaDiff)适配至从想象-NSD基准中重建想象内容。提出一种潜在功能对齐方法,将想象诱发的脑活动映射至预训练模型的条件空间,同时保持其余组件冻结。为缓解有限的匹配想象-感知监督,进一步引入基于检索的增强策略,选择语义相关的NSD感知试验。在4名受试者上,潜在功能对齐持续优于固定预训练基线和体素空间岭对齐基线,且可在多个皮层区域实现高于随机水平的解码。结果表明,从感知中学到的语义结构可被用于稳定并提升分布外条件下的视觉想象解码性能。

原文摘要 · Abstract (English)

Recent progress in visual brain decoding from fMRI has been enabled by large-scale datasets such as the Natural Scenes Dataset (NSD) and powerful diffusion-based generative models. While current pipelines are primarily optimized for perception, their performance under mental-imagery remains less well understood. In this work, we study how a state-of-the-art (SOTA) perception decoder (DynaDiff) can be adapted to reconstruct imagined content from the Imagery-NSD benchmark. We propose a latent functional alignment approach that maps imagery-evoked activity into the pretrained model's conditioning space, while keeping the remaining components frozen. To mitigate the limited amount of matched imagery-perception supervision, we further introduce a retrieval-based augmentation strategy that selects semantically related NSD perception trials. Across four subjects, latent functional alignment consistently improves high-level semantic reconstruction metrics relative to the frozen pretrained baseline and a voxel-space ridge alignment baseline, and enables above-chance decoding from multiple cortical regions. These results suggest that semantic structure learned from perception can be leveraged to stabilize and improve visual imagery decoding under out-of-distribution conditions.

脑解码想象重建扩散模型fMRI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。