用大模型扩写短文本，再用小模型高效建模主题。
Enhancing Short-Text Topic Modeling with LLM-Driven Context Expansion and Prefix-Tuned VAEs
- 先用大模型扩展短文本内容，增强语义信息。
- 通过前缀调优+变分自编码器，在稀疏数据上提升主题质量。
- 适合低资源场景，尤其对短文本主题建模效果显著。
主题建模是挖掘文档集合中隐藏主题的强大技术，但传统方法依赖充分的词共现，而短文本中常缺乏此类统计信号，导致现有概率或神经方法难以提取有意义模式，生成不连贯的主题。为此，我们提出一种新方法：利用大语言模型（LLM）将短文本扩展为更详细的序列，再结合前缀调优的变分自编码器（VAE）进行主题建模。该方法在极端数据稀疏的真实数据集上表现优异，显著优于当前最先进主题模型，有效提升了短文本主题建模的准确性和一致性。
原文摘要 · Abstract (English)
Topic modeling is a powerful technique for uncovering hidden themes within a collection of documents. However, the effectiveness of traditional topic models often relies on sufficient word co-occurrence, which is lacking in short texts. Therefore, existing approaches, whether probabilistic or neural, frequently struggle to extract meaningful patterns from such data, resulting in incoherent topics. To address this challenge, we propose a novel approach that leverages large language models (LLMs) to extend short texts into more detailed sequences before applying topic modeling. To further improve the efficiency and solve the problem of semantic inconsistency from LLM-generated texts, we propose to use prefix tuning to train a smaller language model coupled with a variational autoencoder for short-text topic modeling. Our method significantly improves short-text topic modeling performance, as demonstrated by extensive experiments on real-world datasets with extreme data sparsity, outperforming current state-of-the-art topic models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。