用更符合人类感知的图像模型提升脑电解码准确率
Human-Aligned Image Models Improve Visual Decoding from the Brain
- 采用与人类感知对齐的图像编码器映射脑信号
- 相比顶尖方法，图像检索准确率最高提升21%
- 适用于多种脑电设备、模型和受试者，效果稳定
从脑活动解码视觉图像在脑机交互和理解人类感知方面具有重要意义。近期方法通过对齐图像与脑活动的表示空间实现视觉解码。本文引入人类感知对齐的图像编码器，将脑信号映射为图像。我们假设此类模型能更好捕捉快速视觉刺激下常见的感知特征。实验结果支持该假设，在多种脑电架构、图像编码器、对齐方法、受试者及脑成像模态下，相较于当前最优方法，图像检索准确率最高提升21%。
原文摘要 · Abstract (English)
Decoding visual images from brain activity has significant potential for advancing brain-computer interaction and enhancing the understanding of human perception. Recent approaches align the representation spaces of images and brain activity to enable visual decoding. In this paper, we introduce the use of human-aligned image encoders to map brain signals to images. We hypothesize that these models more effectively capture perceptual attributes associated with the rapid visual stimuli presentations commonly used in visual brain data recording experiments. Our empirical results support this hypothesis, demonstrating that this simple modification improves image retrieval accuracy by up to 21% compared to state-of-the-art methods. Comprehensive experiments confirm consistent performance improvements across diverse EEG architectures, image encoders, alignment methods, participants, and brain imaging modalities
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。