提出新方法TAMLEC,高效完成海量标签文档补全任务。
Extreme Multi-label Completion for Semantic Document Labelling with Taxonomy-Aware Parallel Learning
- 将标签分组为层级路径相关子任务,动态共享特征提升学习效率
- 在真实数据集上超越现有方法,尤其在少样本场景表现突出
- 适合标签体系复杂、标注数据稀少的文档智能标注场景
极端多标签补全(XMLCo)旨在预测文档中缺失的标签。该任务因标签数量庞大(通常超万级)而极具挑战性,且训练数据中标注文档极少。许多方法利用标签间的层次关系(即分类树)来提升性能。本文提出TAMLEC(面向极端多标签补全的层次感知多任务学习),将问题分解为多个与分类树路径对齐的子任务,采用动态并行特征共享机制,在任务间共享部分模型参数但保留任务特异性。推理时,根据文档已有标签选择对应任务进行缺失标签预测。TAMLEC基于改进的Transformer架构,对由任务诱导的弱半格结构输出有序标签序列。实验表明,TAMLEC在多个真实数据集上优于现有最优方法,尤其在少样本新增标签场景下表现卓越。
原文摘要 · Abstract (English)
In Extreme Multi Label Completion (XMLCo), the objective is to predict the missing labels of a collection of documents. Together with XML Classification, XMLCo is arguably one of the most challenging document classification tasks, as the very high number of labels (at least ten of thousands) is generally very large compared to the number of available labelled documents in the training dataset. Such a task is often accompanied by a taxonomy that encodes the labels organic relationships, and many methods have been proposed to leverage this hierarchy to improve the results of XMLCo algorithms. In this paper, we propose a new approach to this problem, TAMLEC (Taxonomy-Aware Multi-task Learning for Extreme multi-label Completion). TAMLEC divides the problem into several Taxonomy-Aware Tasks, i.e. subsets of labels adapted to the hierarchical paths of the taxonomy, and trains on these tasks using a dynamic Parallel Feature sharing approach, where some parts of the model are shared between tasks while others are task-specific. Then, at inference time, TAMLEC uses the labels available in a document to infer the appropriate tasks and to predict missing labels. To achieve this result, TAMLEC uses a modified transformer architecture that predicts ordered sequences of labels on a Weak-Semilattice structure that is naturally induced by the tasks. This approach yields multiple advantages. First, our experiments on real-world datasets show that TAMLEC outperforms state-of-the-art methods for various XMLCo problems. Second, TAMLEC is by construction particularly suited for few-shots XML tasks, where new tasks or labels are introduced with only few examples, and extensive evaluations highlight its strong performance compared to existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。