用大模型动态优化神经主题模型，提升主题可解释性。
Neural Topic Modeling with Large Language Models in the Loop
- 大模型通过最优传输对齐动态修正主题词
- 在保持文档表征质量的同时显著提升主题可读性
- 适配多种神经主题模型，适合需要可解释性的应用
主题建模是自然语言处理中的基础任务，用于发现文本语料中的潜在主题结构。尽管大语言模型（LLM）在主题发现方面展现出潜力，但其直接应用于主题建模存在主题覆盖不全、主题错位和效率低等问题。为此，我们提出一种大模型在环的框架 LLM-ITL，将大语言模型与神经主题模型（NTMs）相结合。在该框架中，全局主题和文档表示由 NTM 学习，同时大模型利用基于最优传输（OT）的对齐目标对主题进行精炼，且精炼程度根据大模型对每组输入词的主题词建议置信度动态调整。该方法具有高度灵活性，可集成到多种现有 NTMs 中，既增强了主题的可解释性，又保留了 NTMs 在学习主题和文档表示方面的高效性。大量实验表明，LLM-ITL 显著提升了 NTMs 的主题可解释性，同时维持了文档表征的质量。代码与数据集已开源。
原文摘要 · Abstract (English)
Topic modeling is a fundamental task in natural language processing, allowing the discovery of latent thematic structures in text corpora. While Large Language Models (LLMs) have demonstrated promising capabilities in topic discovery, their direct application to topic modeling suffers from issues such as incomplete topic coverage, misalignment of topics, and inefficiency. To address these limitations, we propose LLM-ITL, a novel LLM-in-the-loop framework that integrates LLMs with Neural Topic Models (NTMs). In LLM-ITL, global topics and document representations are learned through the NTM. Meanwhile, an LLM refines these topics using an Optimal Transport (OT)-based alignment objective, where the refinement is dynamically adjusted based on the LLM's confidence in suggesting topical words for each set of input words. With the flexibility of being integrated into many existing NTMs, the proposed approach enhances the interpretability of topics while preserving the efficiency of NTMs in learning topics and document representations. Extensive experiments demonstrate that LLM-ITL helps NTMs significantly improve their topic interpretability while maintaining the quality of document representation. Our code and datasets are available at https://github.com/Xiaohao-Yang/LLM-ITL
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。