让深度自编码器像PCA一样生成正交且有序的隐空间。
Orthogonal Dendritic Intrinsic Networks: An Architecture for Significance-Ordered, Orthogonal Latent Spaces
- 通过几何约束直接加入训练目标，实现隐空间正交与方差排序。
- 在合成与真实数据集上均验证了结构化特征学习的有效性。
- 适合需要可解释性降维与特征分解的研究者使用。
主成分分析（PCA）或类似方法具备正交性和方差排序特性，但这些特性在深度自编码器中很少实现。本文提出新型自编码器架构ODIN（Orthogonal Dendritic Intrinsic Network），在完全非线性条件下恢复类PCA的隐空间结构。通过将一组几何约束直接嵌入训练目标，ODIN促使各隐维度相互正交并按解释方差排序，既保留深度网络的表达能力，又实现类似PCA的可解释分解。我们为这些约束提供了理论依据，并证明其与标准编码器-解码器框架兼容。在合成与真实数据集上的实证结果表明，该方法为可解释、结构化的特征学习与降维提供了原则性路径。
原文摘要 · Abstract (English)
Principal Component Analysis or PCA-like properties (orthogonality, variance ranking) are seldom realized in deep autoencoder architectures. In this work, we present ODIN (Orthogonal Dendritic Intrinsic Network), a novel autoencoder architecture that recovers PCA-like latent structure in a fully non-linear regime. By incorporating a set of geometric constraints directly into the training objective, ODIN encourages latent dimensions to be mutually orthogonal and ordered by explained variance, mirroring the interpretable decomposition of PCA while retaining the expressive power of deep networks. We provide theoretical grounding for these constraints and demonstrate their compatibility with standard encoder-decoder frameworks. We also establish empirical results for both synthetic and real world datasets, establishing a principled path toward interpretable, structured feature learning and dimensionality reduction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。