用Transformer实现可扩展的贝叶斯聚类，自动估计簇数与分配。
Transformers can do Bayesian Clustering
- 基于Transformer的Cluster-PFN模型，从合成数据学习后验分布。
- 在真实基因组数据上，缺失率高时仍优于插补方法，且速度更快。
- 无需人工调参，可处理含缺失值的复杂先验，适合大规模数据。
贝叶斯聚类能捕捉不确定性，但大规模计算成本高。真实数据常含缺失值，简单插补忽略不确定性，导致效果不佳。本文提出Cluster-PFN，一种基于Transformer的模型，扩展了先验-数据拟合网络（PFNs）用于无监督贝叶斯聚类。该模型在有限高斯混合模型（GMM）先验生成的合成数据上训练，可同时估计簇数和簇分配的后验分布。相比AIC、BIC及变分推断（VI）等手工模型选择方法，其簇数估计更准确，聚类质量媲美VI，且速度快两个数量级。在包含缺失数据的复杂先验下训练，对真实基因组数据在高缺失率场景下表现优于基于插补的基线方法。结果表明，Cluster-PFN提供了可扩展且灵活的贝叶斯聚类方案。
原文摘要 · Abstract (English)
Bayesian clustering accounts for uncertainty but is computationally demanding at scale. Furthermore, real-world datasets often contain missing values, and simple imputation ignores the associated uncertainty, resulting in suboptimal results. We present Cluster-PFN, a Transformer-based model that extends Prior-Data Fitted Networks (PFNs) to unsupervised Bayesian clustering. Trained entirely on synthetic datasets generated from a finite Gaussian Mixture Model (GMM) prior, Cluster-PFN learns to estimate the posterior distribution over both the number of clusters and the cluster assignments. Our method estimates the number of clusters more accurately than handcrafted model selection procedures such as AIC, BIC and Variational Inference (VI), and achieves clustering quality competitive with VI while being orders of magnitude faster. Cluster-PFN can be trained on complex priors that include missing data, outperforming imputation-based baselines on real-world genomic datasets, at high missingness. These results show that the Cluster-PFN can provide scalable and flexible Bayesian clustering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。