提出递归KL散度优化框架，让表示学习更高效稳定。
Recursive KL Divergence Optimization: A Dynamic Framework for Representation Learning
- 将表示学习看作局部条件分布间递归散度对齐过程。
- 在三个数据集上损失降低约30%，计算资源减少60%-80%。
- 适合追求高效、低资源消耗的表示学习应用。
我们通过将现代表示学习目标重新构想为局部条件分布间的递归散度对齐过程，提出了一种通用化框架。尽管近期方法如信息对比学习（I-Con）通过固定邻域条件下的KL散度统一多种学习范式，但其忽略了学习过程中固有的递归结构。本文提出递归KL散度优化（RKDO），将表示学习视为数据邻域间KL散度动态演化的过程。该形式将对比聚类与降维方法视为静态切片，同时开辟了模型稳定性与局部自适应的新路径。实验表明，RKDO在三个不同数据集上相比静态方法损失值降低约30%，实现相当结果所需计算资源减少60%至80%。这表明RKDO的递归更新机制为表示学习提供了根本更优的优化空间，对资源受限场景具有重要意义。
原文摘要 · Abstract (English)
We propose a generalization of modern representation learning objectives by reframing them as recursive divergence alignment processes over localized conditional distributions While recent frameworks like Information Contrastive Learning I-Con unify multiple learning paradigms through KL divergence between fixed neighborhood conditionals we argue this view underplays a crucial recursive structure inherent in the learning process. We introduce Recursive KL Divergence Optimization RKDO a dynamic formalism where representation learning is framed as the evolution of KL divergences across data neighborhoods. This formulation captures contrastive clustering and dimensionality reduction methods as static slices while offering a new path to model stability and local adaptation. Our experiments demonstrate that RKDO offers dual efficiency advantages approximately 30 percent lower loss values compared to static approaches across three different datasets and 60 to 80 percent reduction in computational resources needed to achieve comparable results. This suggests that RKDOs recursive updating mechanism provides a fundamentally more efficient optimization landscape for representation learning with significant implications for resource constrained applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。