用知识图谱增强零样本多标签分类,发现大模型无需额外图谱提升
Knowledge Graph-Enhanced Zero-Shot Topic Classification: A Multi-Strategy Comparative Study
- 基于文档构建知识图谱,结合多种分类策略进行零样本多标签分类
- 小模型受益于图谱增强,大模型则因预训练已含足够关系信息而表现下降
- 关键词增强法最优,自一致性解码虽耗时五倍却未带来性能提升
无标注训练数据的多标签主题分类极具挑战性,尤其当文档包含复杂关系信息时。本文提出一种零样本多标签主题分类框架,并系统研究了文档级知识图谱增强对性能的影响。基础框架包含四种变体:仅基于文章、关键词增强、以及这两类的自一致性解码版本。随后,为每种变体引入基于输入文档提取的个性化知识图谱(通过类似KGGen的三元组抽取流程生成)。在15个大语言模型和8个跨领域多标签数据集上测试全部八种方法。结果显示,关键词增强方法(AK)表现最佳,其中6个大语言模型超越句向量编码器基线。知识图谱增强对小模型有正面影响,但对大模型产生负面影响,表明大模型在预训练中已掌握充分的关系信息。此外,自一致性解码在所有实验中均未提升性能,且计算成本增加约五倍。
原文摘要 · Abstract (English)
Multi-label topic classification without labeled training data is a challenging task, specially when documents contain complex relational information. We present a zero-shot multi-label topic classification framework and systematically investigate how per-article knowledge graph augmentation affects its performance. The base framework classifies topics in documents without labeled training data and has four variants: article-only classification, keyword-enhanced classification, and self-consistency decoding variants of both. Then, we augment each base variant with per article knowledge graph. This graph is extracted from the input document through a pipeline similar to KGGen based on subject-predicate-object triples. We test all eight methods, four base and four graph augmented on fifteen LLMs and eight multi-label datasets across different domains. For the base framework, keyword-enhanced classification (AK) is the best performing method, and six out of fifteen LLMs surpass the sentence-encoder baseline. Graph augmentation has positive and negative impacts on small and large models, respectively. This shows that larger models already contain enough relational information from pretraining. Furthermore, the self-consistency decoding variant does not show performance improvements in any experiment while increasing computation costs about fivefold.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。