用少量已标注样本引导大模型一键聚类,省去调参且精度更高。
Clustering by Attention: Leveraging Prior Fitted Transformers for Data Partitioning
- 用预训练的PFN模型计算样本间注意力,一次前向传播完成聚类
- 仅需少量预标注样本,即可在多个数据集上超越现有最佳方法
- 适合需要快速部署、避免调参的大规模无监督场景
聚类是机器学习中的核心任务,广泛应用于数据挖掘与模式识别。然而其无监督特性带来诸多挑战:现有算法常需精细调参、计算复杂度高、可解释性差或准确率不足,尤其在大规模数据上表现不佳。本文提出一种基于元学习的新型聚类方法,无需参数优化即可实现优于当前最优技术的精度。该方法利用少量已聚类样本,通过预训练的先验数据拟合变换器网络(PFN)在单次前向传播中完成全数据集聚类。具体而言,模型计算已聚类样本与未聚类样本间的注意力关系,基于学习到的关联推断整体聚类分配。理论与实证均表明,仅需少数预标注样本,模型即可泛化至整个数据集并实现精准聚类。在多个具有挑战性的基准数据集上,即使无预标注样本,本方法仍能有效处理分离良好的数据;当提供少量标注样本时,性能显著提升。结果表明,该方法在准确性和可扩展性上均优于当前先进水平,是一种极具前景的聚类替代方案。
原文摘要 · Abstract (English)
Clustering is a core task in machine learning with wide-ranging applications in data mining and pattern recognition. However, its unsupervised nature makes it inherently challenging. Many existing clustering algorithms suffer from critical limitations: they often require careful parameter tuning, exhibit high computational complexity, lack interpretability, or yield suboptimal accuracy, especially when applied to large-scale datasets. In this paper, we introduce a novel clustering approach based on meta-learning. Our approach eliminates the need for parameter optimization while achieving accuracy that outperforms state-of-the-art clustering techniques. The proposed technique leverages a few pre-clustered samples to guide the clustering process for the entire dataset in a single forward pass. Specifically, we employ a pre-trained Prior-Data Fitted Transformer Network (PFN) to perform clustering. The algorithm computes attention between the pre-clustered samples and the unclustered samples, allowing it to infer cluster assignments for the entire dataset based on the learned relation. We theoretically and empirically demonstrate that, given just a few pre-clustered examples, the model can generalize to accurately cluster the rest of the dataset. Experiments on challenging benchmark datasets show that our approach can successfully cluster well-separated data without any pre-clustered samples, and significantly improves performance when a few clustered samples are provided. We show that our approach is superior to the state-of-the-art techniques. These results highlight the effectiveness and scalability of our approach, positioning it as a promising alternative to existing clustering techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。