arXiv:2507.16490cs.CLcs.LG2025-07

融合语言模型与主题模型特征,发现主题模型反而降低文本分类效果。

Combining Language and Topic Models for Hierarchical Text Classification

  • 用预训练语言模型和主题模型分别提取文本特征
  • 实验显示仅用语言模型特征性能更优,主题模型特征反而拖累结果
  • 适用于验证特征组合有效性或研究多模态特征融合的场景

层次化文本分类(HTC)是将文档归入预定义结构化类别体系的任务。近期方法尝试将层次类结构信息与预训练语言模型(PLM)的语义理解能力结合以提升性能。同时,将主题模型与PLM结合用于提取文本特征,在多标签分类中已被证明有效。其原理在于:PLM捕捉细粒度上下文语义,而主题模型提供全局文档集合的高层表示。本文提出一种方法,使用PLM与主题模型提取文本特征,并输入分类模型。具体流程为:特征经独立卷积层处理后融合,再通过标签注意力机制生成每个类别的专属文档表示。我们在三个HTC基准数据集上进行充分实验,结果显示:使用主题模型提取的特征通常会降低分类性能,相较仅使用PLM特征而言。该结论挑战了以往认为主题模型特征有益的普遍假设,表明在文本分类任务中,不应默认主题模型特征具有增益作用。

原文摘要 · Abstract (English)

Hierarchical text classification (HTC) is a natural language processing task which has the objective of categorising text documents into a set of classes from a predefined structured class hierarchy. Recent HTC approaches use various techniques to incorporate the hierarchical class structure information with the natural language understanding capabilities of pre-trained language models (PLMs) to improve classification performance. Furthermore, using topic models along with PLMs to extract features from text documents has been shown to be an effective approach for multi-label text classification tasks. The rationale behind the combination of these feature extractor models is that the PLM captures the finer-grained contextual and semantic information while the topic model obtains high-level representations which consider the corpus of documents as a whole. In this paper, we use a HTC approach which uses a PLM and a topic model to extract features from text documents which are used to train a classification model. Our objective is to determine whether the combination of the features extracted from the two models is beneficial to HTC performance in general. In our approach, the extracted features are passed through separate convolutional layers whose outputs are combined and passed to a label-wise attention mechanisms which obtains label-specific document representations by weighing the most important features for each class separately. We perform comprehensive experiments on three HTC benchmark datasets and show that using the features extracted from the topic model generally decreases classification performance compared to only using the features obtained by the PLM. In contrast to previous work, this shows that the incorporation of features extracted from topic models for text classification tasks should not be assumed beneficial.

层次分类特征融合模型对比

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。