通过噪声正则化防止DCCA模型坍塌，提升多视图学习稳定性。
Preventing Model Collapse in Deep Canonical Correlation Analysis by Noise Regularization
- 引入噪声正则化，使网络具备相关性不变特性，防止模型坍塌。
- 在合成与真实数据集上均稳定优于基线方法，性能更可靠。
- 适用于DCCA及其变体，适合需要稳健表示学习的研究者。
多视图表示学习旨在从多视角数据中学习对象的统一表示。深度典型相关分析（DCCA）及其变体具有简洁的公式和先进的性能表现。然而，通过大量实验发现，随着训练进行，DCCA类方法会出现性能骤降的现象，即模型坍塌问题，这使得难以确定早停时机，阻碍了其广泛应用。为此，本文提出NR-DCCA，引入一种新型噪声正则化方法以防止模型坍塌。理论分析表明，相关性不变性是防止模型坍塌的关键，而噪声正则化强制神经网络具备该性质。此外，还构建了一个可调控共性与互补信息的合成数据生成框架，用于全面比较多视图学习方法。实验显示，所提NR-DCCA在合成与真实数据集上均显著且稳定优于基线方法，且该噪声正则化策略可推广至其他DCCA变体如DGCCA。
原文摘要 · Abstract (English)
Multi-View Representation Learning (MVRL) aims to learn a unified representation of an object from multi-view data. Deep Canonical Correlation Analysis (DCCA) and its variants share simple formulations and demonstrate state-of-the-art performance. However, with extensive experiments, we observe the issue of model collapse, {\em i.e.}, the performance of DCCA-based methods will drop drastically when training proceeds. The model collapse issue could significantly hinder the wide adoption of DCCA-based methods because it is challenging to decide when to early stop. To this end, we develop NR-DCCA, which is equipped with a novel noise regularization approach to prevent model collapse. Theoretical analysis shows that the Correlation Invariant Property is the key to preventing model collapse, and our noise regularization forces the neural network to possess such a property. A framework to construct synthetic data with different common and complementary information is also developed to compare MVRL methods comprehensively. The developed NR-DCCA outperforms baselines stably and consistently in both synthetic and real-world datasets, and the proposed noise regularization approach can also be generalized to other DCCA-based methods such as DGCCA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。