用信息论量化神经网络表征模糊性，发现连接结构可实现无歧义表征。
Unambiguous Representations in Neural Networks: An Information-Theoretic Approach to Intentionality
- 基于条件熵定义表征模糊性，通过网络连接结构解码内容
- dropout网络解码准确率达100%，普通训练仅38%（随机为10%）
- 能从连接结构还原输入位置，相关系数达0.844，适合意识研究者
表征普遍存在于日常经验中，从字母表示声音到比特串编码数字文件。此类表征需外部解码器赋予意义，而意识体验则不同：一个感知红方块的神经状态不可能同时编码绿三角形的体验。这一内在特性表明，意识表征必须具有非歧义性，而常规表征未必如此。我们利用信息论形式化此直觉，将表征模糊性定义为给定表征R时可能解释I的条件熵H(I|R)。在训练分类MNIST数字的神经网络上实验发现，网络连接关系结构可无歧义地编码表征内容。仅凭连接结构即可实现对输出神经元类别身份的完美识别（dropout训练网络达100%，标准反向传播为38%，随机水平为10%），尽管两者任务表现相同，表明表征模糊性可独立于行为准确性产生。此外，输入神经元的空间位置（与视觉场位置等现象属性相关）可通过连接结构解码，决定系数R²最高达0.844。这些结果为神经系统的表征模糊性提供了量化方法，并证明神经网络可具备理论模型如窄表征主义和整合信息理论所要求的低模糊性表征。
原文摘要 · Abstract (English)
Representations pervade our daily experience, from letters representing sounds to bit strings encoding digital files. While such representations require externally defined decoders to convey meaning, conscious experience is fundamentally different: a neural state corresponding to perceiving a red square cannot alternatively encode the experience of a green triangle. This intrinsic property of consciousness suggests that conscious representations must be unambiguous in a way that conventional representations are not. We formalize this intuition using information theory, defining representational ambiguity as the conditional entropy H(I|R) over possible interpretations I given a representation R. Through experiments on neural networks trained to classify MNIST digits, we demonstrate that relational structures in network connectivity can unambiguously encode representational content. From relational structure alone, we achieve perfect (100%) accuracy for dropout-trained networks and 38% for standard backpropagation (chance: 10%) in identifying output neuron class identity, despite identical task performance, demonstrating that representational ambiguity can arise orthogonally to behavioral accuracy. We further show that spatial position of input neurons, relevant to phenomenal properties like visual field location, can be decoded from network connectivity with R^2 up to 0.844. These results provide a quantitative method for measuring representational ambiguity in neural systems and demonstrate that neural networks can exhibit the low-ambiguity representations posited as necessary (though not sufficient) by theoretical accounts such as narrow representationalism and IIT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。