用大模型生成文本做文档聚类，信息论定义相似性，效果超越以往方法。
Information-Theoretic Generative Clustering of Documents
- 用大模型生成文本，通过KL散度衡量文档间信息相似性。
- 在多个数据集上达到最新最优性能，部分任务提升显著。
- 适合需要高精度聚类与生成式检索的自然语言处理场景。
我们提出一种生成式聚类（GC）方法，通过大语言模型（LLM）生成的文本Y对文档集X进行聚类，而非直接对原始文档X聚类。由于LLM能输出概率分布，可通过KL散度以信息论方式严格定义两篇文档间的相似性。我们还提出一种基于重要性采样的新颖聚类算法。实验表明，该方法在多个数据集上均取得当前最佳性能，显著优于以往所有聚类方法。此外，我们将其应用于生成式文档检索，通过层次聚类索引文档，显著提升了检索准确率。
原文摘要 · Abstract (English)
We present {\em generative clustering} (GC) for clustering a set of documents, $\mathrm{X}$, by using texts $\mathrm{Y}$ generated by large language models (LLMs) instead of by clustering the original documents $\mathrm{X}$. Because LLMs provide probability distributions, the similarity between two documents can be rigorously defined in an information-theoretic manner by the KL divergence. We also propose a natural, novel clustering algorithm by using importance sampling. We show that GC achieves the state-of-the-art performance, outperforming any previous clustering method often by a large margin. Furthermore, we show an application to generative document retrieval in which documents are indexed via hierarchical clustering and our method improves the retrieval accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。