用大模型筛选与聚合,让通用嵌入模型无需微调就能适应新领域意图聚类。
SPILL: Domain-Adaptive Intent Clustering based on Selection and Pooling with Large Language Models
- 以种子句为起点,通过距离筛选候选句,再用大模型判断意图一致性。
- 在多个数据集上优于直接使用嵌入模型,接近需微调的先进方法性能。
- 无需微调即可提升通用嵌入模型的领域适应性,适合快速部署场景。
本文提出基于大语言模型的筛选与聚合方法(SPILL),一种无需微调的直观且领域自适应的意图聚类方法。现有基于嵌入的聚类方法依赖少量标注样本或无监督微调来优化新数据集结果,泛化能力有限。我们目标是使现有嵌入模型在不进行额外微调的情况下,更好地适应新领域数据。受理论推导和模拟结果启发,我们视聚类任务为小规模筛选问题,良好解对应更高聚类性能。方法分两阶段:首先用现有嵌入模型获取每个语句(种子)的嵌入,再以距离度量筛选靠近种子的候选句;由于嵌入模型未针对新数据集优化,在第二阶段,利用大语言模型从候选中进一步筛选与种子意图一致的语句;最后将这些选中的候选句与种子合并,生成精炼后的种子嵌入。实验表明,该方法普遍优于直接使用嵌入模型,且性能媲美需更大模型和微调的前沿方法,体现其高效与强大。结果说明,该方法可在不增加训练成本的前提下提升嵌入模型的领域适应性,同时将聚类视为小规模筛选问题,为用户导向的定制化聚类提供了可能。
原文摘要 · Abstract (English)
In this paper, we propose Selection and Pooling with Large Language Models (SPILL), an intuitive and domain-adaptive method for intent clustering without fine-tuning. Existing embeddings-based clustering methods rely on a few labeled examples or unsupervised fine-tuning to optimize results for each new dataset, which makes them less generalizable to multiple datasets. Our goal is to make these existing embedders more generalizable to new domain datasets without further fine-tuning. Inspired by our theoretical derivation and simulation results on the effectiveness of sampling and pooling techniques, we view the clustering task as a small-scale selection problem. A good solution to this problem is associated with better clustering performance. Accordingly, we propose a two-stage approach: First, for each utterance (referred to as the seed), we derive its embedding using an existing embedder. Then, we apply a distance metric to select a pool of candidates close to the seed. Because the embedder is not optimized for new datasets, in the second stage, we use an LLM to further select utterances from these candidates that share the same intent as the seed. Finally, we pool these selected candidates with the seed to derive a refined embedding for the seed. We found that our method generally outperforms directly using an embedder, and it achieves comparable results to other state-of-the-art studies, even those that use much larger models and require fine-tuning, showing its strength and efficiency. Our results indicate that our method enables existing embedders to be further improved without additional fine-tuning, making them more adaptable to new domain datasets. Additionally, viewing the clustering task as a small-scale selection problem gives the potential of using LLMs to customize clustering tasks according to the user's goals.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。