DSM网络因熵约束导致特征表达力下降，引发频谱坍缩。
The Homogeneity Trap: Spectral Collapse in Doubly-Stochastic Deep Networks
- 通过分析熵最大投影机制，揭示其使混合算子趋向均匀中心
- 发现次主导奇异值σ₂随有效深度增加而被抑制，特征变换受限
- 证明层归一化在噪声主导下无法缓解坍缩，适合关注稳定性的研究者
双重随机矩阵（DSM）在保持结构的深度网络中日益广泛应用，如最优传输层和基于Sinkhorn的注意力机制，以增强数值稳定性和概率可解释性。本文识别出此类约束固有的关键频谱退化现象，称为同质性陷阱。我们证明，典型的Sinkhorn投影带来的最大熵偏差，会驱动混合算子趋向于均匀重心，从而抑制次主导奇异值σ₂，滤除高频特征成分。我们推导出σ₂与网络有效深度的频谱界，表明高熵约束限制了特征变换的有效感受野。此外，我们正式证明，在频谱滤波降低信噪比（SNR）至临界阈值以下时，层归一化无法缓解噪声主导下的几何结构崩溃。研究揭示了熵稳定性与频谱表达力之间的根本权衡。
原文摘要 · Abstract (English)
Doubly-stochastic matrices (DSM) are increasingly utilized in structure-preserving deep architectures -- such as Optimal Transport layers and Sinkhorn-based attention -- to enforce numerical stability and probabilistic interpretability. In this work, we identify a critical spectral degradation phenomenon inherent to these constraints, termed the Homogeneity Trap. We demonstrate that the maximum-entropy bias, typical of Sinkhorn-based projections, drives the mixing operator towards the uniform barycenter, thereby suppressing the subdominant singular value σ_2 and filtering out high-frequency feature components. We derive a spectral bound linking σ_2 to the network's effective depth, showing that high-entropy constraints restrict feature transformation to a shallow effective receptive field. Furthermore, we formally demonstrate that Layer Normalization fails to mitigate this collapse in noise-dominated regimes; specifically, when spectral filtering degrades the Signal-to-Noise Ratio (SNR) below a critical threshold, geometric structure is irreversibly lost to noise-induced orthogonal collapse. Our findings highlight a fundamental trade-off between entropic stability and spectral expressivity in DSM-constrained networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。