arXiv:2505.12552eess.IVcs.AI2025-05中稿 · the British Machin…

通过自适应筛选频段,提升脑影像还原图像的清晰度与可解释性。

FreqSelect: Frequency-Aware fMRI-to-Image Reconstruction

  • 在编码前动态筛选对脑活动预测性强的频段,抑制无效噪声。
  • 在Natural Scenes数据集上显著提升低频与高频重建质量。
  • 无需额外标注,可适配多种模型,助力神经科学分析。

从功能磁共振成像(fMRI)数据中重构自然图像仍是自然解码的核心挑战,源于视觉刺激的丰富性与fMRI信号噪声大、分辨率低之间的不匹配。尽管近期两阶段模型(结合深度变分自编码器与扩散模型)已取得进展,但它们对输入的所有空间频率成分一视同仁,迫使模型同时提取语义特征并压制无关噪声,限制了效果。本文提出轻量级自适应模块FreqSelect,可在编码前选择性过滤空间频率带。通过动态增强对脑活动最具预测性的频率,抑制无信息频率,该模块充当图像特征与自然数据间的语义感知闸门。它可无缝集成至标准的深度VAE-扩散模型流程中,无需额外监督。在Natural Scenes数据集上的评估表明,FreqSelect在低频与高频指标上均持续提升重建质量。此外,学习到的频率选择模式为不同视觉频率在大脑中的表征提供了可解释洞察。方法具有跨被试与场景的泛化能力,有望拓展至其他神经成像模态,为提升解码精度与神经科学可解释性提供系统性方案。

原文摘要 · Abstract (English)

Reconstructing natural images from functional magnetic resonance imaging (fMRI) data remains a core challenge in natural decoding due to the mismatch between the richness of visual stimuli and the noisy, low resolution nature of fMRI signals. While recent two-stage models, combining deep variational autoencoders (VAEs) with diffusion models, have advanced this task, they treat all spatial-frequency components of the input equally. This uniform treatment forces the model to extract meaning features and suppress irrelevant noise simultaneously, limiting its effectiveness. We introduce FreqSelect, a lightweight, adaptive module that selectively filters spatial-frequency bands before encoding. By dynamically emphasizing frequencies that are most predictive of brain activity and suppressing those that are uninformative, FreqSelect acts as a content-aware gate between image features and natural data. It integrates seamlessly into standard very deep VAE-diffusion pipelines and requires no additional supervision. Evaluated on the Natural Scenes dataset, FreqSelect consistently improves reconstruction quality across both low- and high-level metrics. Beyond performance gains, the learned frequency-selection patterns offer interpretable insights into how different visual frequencies are represented in the brain. Our method generalizes across subjects and scenes, and holds promise for extension to other neuroimaging modalities, offering a principled approach to enhancing both decoding accuracy and neuroscientific interpretability.

脑机接口图像重建频域处理可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。