解析UMAP的引力与斥力机制，揭示聚类形成原理
The Shape of Attraction in UMAP: Exploring the Embedding Forces in Dimensionality Reduction
- 通过分析引力与斥力作用，揭示低维嵌入中聚类形态的成因
- 斥力控制簇边界和簇间距离，引力在低维映射中具双重效应
- 改进引力机制可提升随机初始化下的聚类一致性
统一流形近似与投影（UMAP）是当前最流行的邻域嵌入方法之一。该方法根据高维空间中的相似性采样点对，并在低维嵌入中对它们的坐标施加吸引与排斥力。本文分析这些力的作用，揭示其对聚类形成与可视化的影响，并与同类方法对比。斥力强调差异，控制簇边界与簇间距离；引力则更微妙，点间的吸引力在低维映射中可能同时表现为吸引或排斥。这解释了学习率退火的必要性，并提示应区别对待吸引与排斥项。此外，通过调整引力机制，我们提升了随机初始化下聚类形成的稳定性。整体上，本研究为UMAP及相关嵌入方法提供了机制层面的理解。
原文摘要 · Abstract (English)
Uniform manifold approximation and projection (UMAP) is among the most popular neighbor embedding methods. The method samples pairs of point indices according to similarities in the high-dimensional space, and applies attractive and repulsive forces to their coordinates in the low-dimensional embedding. In this paper, we analyze the forces to reveal their effects on cluster formations and visualization, and compare UMAP to its contemporaries. Repulsion emphasizes differences, controlling cluster boundaries and inter-cluster distance. Attraction is more subtle, as attractive tension between points can manifest simultaneously as attraction and repulsion in the lower-dimensional mapping. This explains the need for learning rate annealing and motivates the different treatments between attractive and repulsive terms. Moreover, by modifying attraction, we improve the consistency of cluster formation under random initialization. Overall, our analysis provides a mechanistic understanding of UMAP and related embedding methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。