arXiv:2607.15835cs.LG2026-07

Big-means++高效全局搜索,让海量数据聚类不再困于局部最优。

Data-Native Global Optimization for Big Data K-means Clustering

论文配图:Data-Native Global Optimization for Big Data K-means Clustering
图 1 · 摘自论文原文
  • 通过采样构建代理景观,以样本变化驱动全局搜索。
  • 在22个数据集上优于11种算法,收敛快且抗不平衡性强。
  • 适合超大数据量、对精度要求高的工业级聚类任务。

大规模数据聚类仍具挑战:基于K均值的最小平方和聚类(MSSC)问题是NP难的,现有方法或陷入劣质局部最优,或需代价高昂的元启发式混合。本文针对任意高维数据——特征空间固定但观测数可无限增长,算法仅访问有限随机采样——提出Big-means++,通过优化输入来实现大尺度数据聚类的可扩展性与全局搜索质量。该算法将局部K均值优化整合为数据原生的全局搜索机制,不直接优化全数据MSSC目标,而是遍历由采样诱导的代理景观。每个样本定义一个带扰动局部最优结构的经验MSSC近似,使样本间差异成为全局搜索驱动力。不同于旧法的流动候选策略,Big-means++通过新采样上的K均值精炼传递中心状态,无需回溯最佳解,提升迁移能力并稳定高质量配置。新增几何级变化采样大小的抖动机制,拓宽多尺度代理景观探索,缓解聚类不平衡,提升解质量。采用竞争式多智能体系统异步探索独立采样景观,将多样化随机轨迹转化为集体搜索智能。自动收敛检测在各智能体获得高质量解后停止搜索,防止性能退化,同时提供普适的速度-质量控制。在22个数据集上对比11种算法的实验表明,Big-means++具有显著有效性、效率与鲁棒性。

原文摘要 · Abstract (English)

Big data clustering remains challenging: the Minimum Sum-of-Squares Clustering (MSSC) problem underlying K-means is NP-hard, and existing methods either reach poor local minima or require prohibitive metaheuristic hybrids. We target arbitrarily tall data: a fixed feature space may contain arbitrarily many, possibly infinitely many, observations, while the algorithm accesses only finite random samples. We propose Big-means++, an algorithm achieving scalability and global-search quality by curating inputs to MSSC optimization on big data. It orchestrates local K-means refinements into a data-native global search for big data clustering. Rather than optimizing the full-data MSSC objective, Big-means++ traverses sample-induced surrogate landscapes. Each sample defines a distinct empirical MSSC approximation with a perturbed local-optimum structure, turning sample-to-sample variation into a global-search mechanism. Unlike Big-means, a flowing-incumbent strategy propagates centroid state across empirical landscapes through K-means refinements on fresh samples without rollback to a best-so-far solution. This increases mobility and favors stable, high-quality configurations across approximations of the full-data structure. A new shaking mechanism varies sample size geometrically, broadening the surrogate landscapes explored across resolution scales, accounting for cluster imbalance, and improving solution quality. A competitive multi-agent system asynchronously explores independent sampled landscapes, transforming diverse stochastic trajectories into collective search intelligence. Automatic convergence detection stops each agent after attaining a high-quality solution but before further search risks degrading it, while providing a universal speed-quality control. Experiments on 22 datasets against 11 competing algorithms demonstrate the effectiveness, efficiency, and robustness of Big-means++.

聚类大数据全局优化算法设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。