揭示集成聚类泛化能力的理论边界,指导更优算法设计。
Generalization Performance of Ensemble Clustering: From Theory to Algorithm
- 从理论上推导出泛化误差与过拟合风险的收敛速率
- 证明当基聚类数远大于样本对数时,集成聚类具有一致性
- 提出加权集成策略,强调基聚类偏差小且差异大才能性能更好
集成聚类在实践中表现优异,但其理论基础仍不充分。本文研究集成聚类的泛化性能,聚焦泛化误差、过拟合风险和一致性。我们推导出泛化误差与过拟合风险界均达到 $\/mathcal{O}(\ ext{sqrt}{\rac{\ ext{log} n}{m}} + \rac{1}{\ ext{sqrt}{n}})$ 的收敛速率,其中 $n$ 和 $m$ 分别为样本数与基聚类数。基于此,证明当 $m,n\to \infty$ 且 $m\gg \text{log} n$ 时,集成聚类具有一致性。由于实际中 $n$ 与 $m$ 有限,泛化误差无法归零,因此通过给有限基聚类赋予权重,最小化经验平均聚类与其期望之间的误差。理论表明,提升聚类性能需最小化基聚类与期望的偏差(bias),并最大化各基聚类间的差异性(diversity)。进一步推导出最大化多样性近似于鲁棒(极小极大)优化模型。最后,基于该理论构建新算法,在10个数据集上对NMI、ARI、Purity分别取得6.1%、7.3%、6.0%的平均提升。代码已开源:https://github.com/xuz2019/GPEC。
原文摘要 · Abstract (English)
Ensemble clustering has demonstrated great success in practice; however, its theoretical foundations remain underexplored. This paper examines the generalization performance of ensemble clustering, focusing on generalization error, excess risk and consistency. We derive a convergence rate of generalization error bound and excess risk bound both of $\mathcal{O}(\sqrt{\frac{\log n}{m}}+\frac{1}{\sqrt{n}})$, with $n$ and $m$ being the numbers of samples and base clusterings. Based on this, we prove that when $m$ and $n$ approach infinity and $m$ is significantly larger than log $n$, i.e., $m,n\to \infty, m\gg \log n$, ensemble clustering is consistent. Furthermore, recognizing that $n$ and $m$ are finite in practice, the generalization error cannot be reduced to zero. Thus, by assigning varying weights to finite clusterings, we minimize the error between the empirical average clusterings and their expectation. From this, we theoretically demonstrate that to achieve better clustering performance, we should minimize the deviation (bias) of base clustering from its expectation and maximize the differences (diversity) among various base clusterings. Additionally, we derive that maximizing diversity is nearly equivalent to a robust (min-max) optimization model. Finally, we instantiate our theory to develop a new ensemble clustering algorithm. Compared with SOTA methods, our approach achieves average improvements of 6.1%, 7.3%, and 6.0% on 10 datasets w.r.t. NMI, ARI, and Purity. The code is available at https://github.com/xuz2019/GPEC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。