提出ADM算法，通过对抗机制消除特征间的非线性依赖。
Adversarial Dependence Minimization
- 用对抗网络识别并消除特征维度间的统计依赖
- 在图像分类中提升模型泛化能力，防止自监督学习中的维数坍缩
- 可扩展主成分分析至非线性去相关，适合需要紧凑独立表征的场景
最小冗余表示通常通过最小化特征协方差来学习，但基于协方差的方法无法消除所有依赖关系，因为线性不相关的变量仍可能存在非线性关系。为此，我们提出ADM，一种可微的算法，通过对抗游戏最小化特征维度间的统计依赖：辅助网络识别依赖，编码器则消除它们。我们证明，在全局最优时可实现相互独立，实证验证了收敛性，并研究了三个应用场景：将PCA扩展至非线性去相关、提升图像分类的泛化性能、防止自监督学习中的维数坍缩。通过促进统计独立的表示，ADM为在多种应用中学习更鲁棒、压缩和通用的表示铺平了道路。
原文摘要 · Abstract (English)
Minimally redundant representations are typically learned by minimizing feature covariance. However, covariance-based methods fail to eliminate all dependencies/redundancies, as linearly uncorrelated variables can still exhibit nonlinear relationships. To address this, we introduce ADM, a differentiable algorithm that minimizes statistical dependence between feature dimensions through an adversarial game: auxiliary networks identify dependencies, while the encoder removes them. We prove that mutual independence is achieved at the global optimum, empirically verify convergence, and study three potential applications: extending PCA to nonlinear decorrelation, improving generalization in image classification, and preventing dimensional collapse in self-supervised learning. By promoting statistically independent representations, ADM paves the way for learning more robust, compressed, and generalizable representations across diverse applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。