arXiv:2503.04207cs.CV2025-03CVPR被引 40

用不确定性模糊先验,提升脑信号还原图像的准确性。

Bridging the Vision-Brain Gap with an Uncertainty-Aware Blur Prior

  • 根据脑信号与图像的不匹配程度,动态模糊图像高频细节
  • 零样本脑图检索达50.9%准确率,优于之前方法13.7个百分点
  • 适合脑机接口、神经影像分析等需要高精度重建的研究者

人的脑信号能否忠实反映原始视觉刺激,包括高频细节?尽管人类感知和认知能力可处理与记忆视觉信息,但受限于注意力资源与视觉记忆容量,视觉系统在将刺激转化为脑信号时不可避免地丢失部分信息,形成所谓的「系统差距」。此外,感知动态与采集噪声进一步降低脑信号保真度,称为「随机差距」。当直接对齐脑信号与预训练图像特征时,这些差距使模型难以学习,尤其在配对数据有限的情况下易过拟合、泛化差。为此,我们提出一种简单有效的「不确定性感知模糊先验(UBP)」方法:通过估计配对数据中的不确定性,反映脑信号与视觉刺激间的不匹配程度,并据此动态模糊原始图像的高频细节,减轻不匹配影响,增强对齐效果。该方法在零样本脑到图像检索任务中取得50.9%的top-1准确率和79.7%的top-5准确率,分别超越先前最优方法13.7和9.8个百分点。代码已开源。

原文摘要 · Abstract (English)

Can our brain signals faithfully reflect the original visual stimuli, even including high-frequency details? Although human perceptual and cognitive capacities enable us to process and remember visual information, these abilities are constrained by several factors, such as limited attentional resources and the finite capacity of visual memory. When visual stimuli are processed by human visual system into brain signals, some information is inevitably lost, leading to a discrepancy known as the \textbf{System GAP}. Additionally, perceptual and cognitive dynamics, along with technical noise in signal acquisition, degrade the fidelity of brain signals relative to the visual stimuli, known as the \textbf{Random GAP}. When encoded brain representations are directly aligned with the corresponding pretrained image features, the System GAP and Random GAP between paired data challenge the model, requiring it to bridge these gaps. However, in the context of limited paired data, these gaps are difficult for the model to learn, leading to overfitting and poor generalization to new data. To address these GAPs, we propose a simple yet effective approach called the \textbf{Uncertainty-aware Blur Prior (UBP)}. It estimates the uncertainty within the paired data, reflecting the mismatch between brain signals and visual stimuli. Based on this uncertainty, UBP dynamically blurs the high-frequency details of the original images, reducing the impact of the mismatch and improving alignment. Our method achieves a top-1 accuracy of \textbf{50.9\%} and a top-5 accuracy of \textbf{79.7\%} on the zero-shot brain-to-image retrieval task, surpassing previous state-of-the-art methods by margins of \textbf{13.7\%} and \textbf{9.8\%}, respectively. Code is available at \href{https://github.com/HaitaoWuTJU/Uncertainty-aware-Blur-Prior}{GitHub}.

脑机接口图像重建不确定性建模模糊先验

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。