从信息论角度分析多视图学习的泛化能力,揭示其成功关键。
Towards the Generalization of Multi-view Learning: An Information-theoretical Analysis
- 基于信息论构建多视图学习的泛化界,强调共识与互补信息的重要性。
- 提出可计算的紧致数据依赖界,在插值情形下实现更快收敛速率。
- 适用于研究多视图表示学习泛化机制的研究者,尤其关注理论分析者。
多视图学习因其能有效利用跨视图的一致性与互补性信息,实现数据的全面表征而受到广泛关注。尽管多视图学习发展迅速并取得显著成果,其泛化行为的理论理解仍不充分。本文通过建立多视图学习的信息论泛化界,重点分析多视图重构与分类任务,揭示捕捉多视角一致性和互补性信息对获得最大解耦表征的重要性。结果表明,应用多视图信息瓶颈正则化有助于实现良好的泛化性能。此外,我们在留一法和超采样设定下推导出新型数据依赖界,具备计算可行性且更紧致。在插值区间,进一步建立了快速率泛化界,其收敛速度优于传统平方根界。数值实验显示,真实泛化差距与所推导界之间存在强相关性,涵盖多种学习场景。
原文摘要 · Abstract (English)
Multiview learning has drawn widespread attention for its efficacy in leveraging cross-view consensus and complementarity information to achieve a comprehensive representation of data. While multi-view learning has undergone vigorous development and achieved remarkable success, the theoretical understanding of its generalization behavior remains elusive. This paper aims to bridge this gap by developing information-theoretic generalization bounds for multi-view learning, with a particular focus on multi-view reconstruction and classification tasks. Our bounds underscore the importance of capturing both consensus and complementary information from multiple different views to achieve maximally disentangled representations. These results also indicate that applying the multi-view information bottleneck regularizer is beneficial for satisfactory generalization performance. Additionally, we derive novel data-dependent bounds under both leave-one-out and supersample settings, yielding computational tractable and tighter bounds. In the interpolating regime, we further establish the fast-rate bound for multi-view learning, exhibiting a faster convergence rate compared to conventional square-root bounds. Numerical results indicate a strong correlation between the true generalization gap and the derived bounds across various learning scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。