神经表征分析可能因模型偏见而误判系统机制，影响对大脑和模型的理解。
Representation biases: will we achieve complete understanding by analyzing representations?
- 发现模型学习的表征存在偏差，简单特征被过度表示，复杂特征则较弱且不一致。
- 常见分析方法如PCA、回归和RSA会因表征偏见产生严重误导性结论。
- 适合关注表征分析可靠性、神经科学与机器学习交叉研究的读者。
在神经科学中，常通过分析神经表征来理解系统，越来越多地将神经表征与计算模型内部表征进行对比。然而，机器学习近期研究（Lampinen, 2024）表明，学习到的特征表征可能存在偏见：某些特征被过度表示，而其他特征则表现更弱、更不一致。例如，线性特征比高度非线性的复杂特征更强烈、更一致地被表征。这种偏差可能阻碍通过表征分析实现对系统的完整理解。本文揭示了这些挑战——展示表征偏差如何导致主成分分析（PCA）、回归和表示相似性分析（RSA）等常用方法产生严重偏差的推断。以同态加密为案例，说明表征模式与计算过程之间可能存在强分离。讨论了这些结果对跨系统表征比较及神经科学的启示。
原文摘要 · Abstract (English)
A common approach in neuroscience is to study neural representations as a means to understand a system -- increasingly, by relating the neural representations to the internal representations learned by computational models. However, a recent work in machine learning (Lampinen, 2024) shows that learned feature representations may be biased to over-represent certain features, and represent others more weakly and less-consistently. For example, simple (linear) features may be more strongly and more consistently represented than complex (highly nonlinear) features. These biases could pose challenges for achieving full understanding of a system through representational analysis. In this perspective, we illustrate these challenges -- showing how feature representation biases can lead to strongly biased inferences from common analyses like PCA, regression, and RSA. We also present homomorphic encryption as a simple case study of the potential for strong dissociation between patterns of representation and computation. We discuss the implications of these results for representational comparisons between systems, and for neuroscience more generally.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。