用大模型指导小模型,精准发现细分领域话题
PRISM: LLM-Guided Semantic Clustering for High-Precision Topics
- 用少量大模型标注数据微调句子编码器,构建轻量级主题模型
- 在多个语料库上实现比主流局部主题模型更优的话题分离度
- 适合需要可解释性、本地部署的网络文本细粒度分析场景
本文提出精密感知语义建模(PRISM),一种融合大语言模型丰富表征与潜在语义聚类低成本、可解释优势的结构化主题建模框架。PRISM 使用从目标语料中采样样本并由大模型提供稀疏标签,对句子编码模型进行微调。通过阈值聚类分割嵌入空间,生成在窄域内区分紧密相关话题的聚类结果。在多个语料库上,PRISM 在仅需少量大模型查询的情况下,相比前沿嵌入模型的聚类效果及当前最优局部主题模型,均实现了更高的话题可分性。本工作贡献包括:(i) 构建学生-教师管道,将稀疏的大模型监督蒸馏为轻量级主题发现模型;(ii) 分析采样策略对提升局部几何结构以增强聚类可分性的有效性;(iii) 提出适用于网络规模文本分析的有效方法,使研究者与从业者能以可解释、本地部署的框架追踪线上细微主张与子话题。
原文摘要 · Abstract (English)
In this paper, we propose Precision-Informed Semantic Modeling (PRISM), a structured topic modeling framework combining the benefits of rich representations captured by LLMs with the low cost and interpretability of latent semantic clustering methods. PRISM fine-tunes a sentence encoding model using a sparse set of LLM- provided labels on samples drawn from some corpus of interest. We segment this embedding space with thresholded clustering, yielding clusters that separate closely related topics within some narrow domain. Across multiple corpora, PRISM improves topic separability over state-of-the-art local topic models and even over clustering on large, frontier embedding models while requiring only a small number of LLM queries to train. This work contributes to several research streams by providing (i) a student-teacher pipeline to distill sparse LLM supervision into a lightweight model for topic discovery; (ii) an analysis of the efficacy of sampling strategies to improve local geometry for cluster separability; and (iii) an effective approach for web-scale text analysis, enabling researchers and practitioners to track nuanced claims and subtopics online with an interpretable, locally deployable framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。