提出可解释非线性神经编码模型的新框架,解析大脑视觉处理的层次化非线性特征。
LinBridge: A Learnable Framework for Interpreting Nonlinear Neural Encoding Models
- 基于雅可比矩阵分解非线性映射为线性主成分与样本特异性偏差
- 在CLIP-ViT预测fMRI脑活动时,准确捕捉复杂非线性关系
- 揭示视觉皮层不同层级间非线性程度的差异,适合神经科学与计算认知研究
人工神经网络(ANN)的神经编码将计算表征与脑响应关联,揭示大脑信息处理机制。现有研究多采用线性模型以保证可解释性,但脑响应常具非线性特征。为此,本文提出LinBridge——一种基于雅可比分析的可学习、灵活的非线性编码模型解释框架。该框架假设:从ANN表征到神经响应的非线性映射可分解为近似复杂非线性关系的线性固有成分,以及捕捉样本特异性非线性的映射偏差。雅可比矩阵反映输出对输入的变化率,支持对非线性模型中样本选择性映射的分析。LinBridge采用自监督学习策略,从测试集的雅可比矩阵中提取线性固有成分与非线性映射偏差,从而有效适应多种非线性编码模型。我们在视觉神经编码场景下验证该框架,使用CLIP-ViT生成的计算视觉表征预测功能性磁共振成像(fMRI)记录的脑活动。实验表明:1)由LinBridge提取的线性固有成分能准确反映非线性神经编码模型的复杂映射;2)样本特异性映射偏差揭示了视觉加工层级间非线性变化的差异。本研究为解释非线性神经编码模型提供新工具,并为视觉皮层中层次化非线性分布提供了新证据。
原文摘要 · Abstract (English)
Neural encoding of artificial neural networks (ANNs) links their computational representations to brain responses, offering insights into how the brain processes information. Current studies mostly use linear encoding models for clarity, even though brain responses are often nonlinear. This has sparked interest in developing nonlinear encoding models that are still interpretable. To address this problem, we propose LinBridge, a learnable and flexible framework based on Jacobian analysis for interpreting nonlinear encoding models. LinBridge posits that the nonlinear mapping between ANN representations and neural responses can be factorized into a linear inherent component that approximates the complex nonlinear relationship, and a mapping bias that captures sample-selective nonlinearity. The Jacobian matrix, which reflects output change rates relative to input, enables the analysis of sample-selective mapping in nonlinear models. LinBridge employs a self-supervised learning strategy to extract both the linear inherent component and nonlinear mapping biases from the Jacobian matrices of the test set, allowing it to adapt effectively to various nonlinear encoding models. We validate the LinBridge framework in the scenario of neural visual encoding, using computational visual representations from CLIP-ViT to predict brain activity recorded via functional magnetic resonance imaging (fMRI). Our experimental results demonstrate that: 1) the linear inherent component extracted by LinBridge accurately reflects the complex mappings of nonlinear neural encoding models; 2) the sample-selective mapping bias elucidates the variability of nonlinearity across different levels of the visual processing hierarchy. This study presents a novel tool for interpreting nonlinear neural encoding models and offers fresh evidence about hierarchical nonlinearity distribution in the visual cortex.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。