用神经元最兴奋输入提升脑电解码,小样本下也能还原清晰图像。
MEIcoder: Decoding Visual Stimuli from Neural Activity by Leveraging Most Exciting Inputs
- 利用神经元最兴奋输入(MEI)构建生物启发解码模型
- 仅需1000-2500个神经元和少于1000个样本即可还原高保真图像
- 适合脑机接口与神经科学小样本研究,可推广至人类应用
从神经群体活动解码视觉刺激对理解大脑及脑机接口至关重要。然而,灵长类或人类的生物数据通常稀缺,因高通量记录技术(如双光子成像)难以应用,这制约了深度学习解码方法的发展。为此,我们提出MEIcoder,一种融合神经元特异性最兴奋输入(MEIs)、结构相似性指数损失和对抗训练的生物启发解码方法。该方法在初级视觉皮层(V1)单细胞活动中实现了最先进的视觉刺激重建性能,尤其在小数据集和少量记录神经元下表现优异。消融实验表明,MEIs是性能主要驱动因素;缩放实验显示,仅需1000–2500个神经元和少于1000个训练样本即可重建出自然逼真的图像。我们还提出了一个包含超过16万样本的统一基准,以推动未来研究。结果证明早期视觉系统可靠解码的可行性,并为神经科学与神经工程提供实用洞见。
原文摘要 · Abstract (English)
Decoding visual stimuli from neural population activity is crucial for understanding the brain and for applications in brain-machine interfaces. However, such biological data is often scarce, particularly in primates or humans, where high-throughput recording techniques, such as two-photon imaging, remain challenging or impossible to apply. This, in turn, poses a challenge for deep learning decoding techniques. To overcome this, we introduce MEIcoder, a biologically informed decoding method that leverages neuron-specific most exciting inputs (MEIs), a structural similarity index measure loss, and adversarial training. MEIcoder achieves state-of-the-art performance in reconstructing visual stimuli from single-cell activity in primary visual cortex (V1), especially excelling on small datasets with fewer recorded neurons. Using ablation studies, we demonstrate that MEIs are the main drivers of the performance, and in scaling experiments, we show that MEIcoder can reconstruct high-fidelity natural-looking images from as few as 1,000-2,500 neurons and less than 1,000 training data points. We also propose a unified benchmark with over 160,000 samples to foster future research. Our results demonstrate the feasibility of reliable decoding in early visual system and provide practical insights for neuroscience and neuroengineering applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。