用自回归解码器实现高效层次文本分类,不依赖标签语义和图编码器。
A Transformer-based Autoregressive Decoder Architecture for Hierarchical Text Classification
- 基于RoBERTa编码器+双层自回归解码器,直接生成标签序列。
- 在三个数据集上性能媲美最先进方法,推理速度提升2倍。
- 无需标签语义或层次图编码,适合快速替换新编码器的场景。
当前层次文本分类(HTC)方法依赖预训练Transformer模型,结合标签语义与图编码器处理标签层级结构。本文提出一种仅使用现成RoBERTa作为编码器、自定义两层自回归解码器生成分类结果的新型分类器RADAr。与现有方法不同,RADAr的编码器不显式编码标签层级,解码器仅依赖训练中观察到的标签序列。在三个基准数据集上的实验表明,RADAr性能可媲美当前最优方法,且训练与推理时间更短。当将标签序列按从子节点到父节点顺序组织时,性能优于反向顺序。实验显示,无需标签语义或显式的层次图编码器。该设计具有重要实用价值:减少模型依赖,推理速度提升2倍;同时支持未来研究者灵活更换编码器模块。代码已开源。
原文摘要 · Abstract (English)
Recent approaches in hierarchical text classification (HTC) rely on the capabilities of a pre-trained transformer model and exploit the label semantics and a graph encoder for the label hierarchy. In this paper, we introduce an effective hierarchical text classifier RADAr (Transformer-based Autoregressive Decoder Architecture) that is based only on an off-the-shelf RoBERTa transformer to process the input and a custom autoregressive decoder with two decoder layers for generating the classification output. Thus, unlike existing approaches for HTC, the encoder of RADAr has no explicit encoding of the label hierarchy and the decoder solely relies on the label sequences of the samples observed during training. We demonstrate on three benchmark datasets that RADAr achieves results competitive to the state of the art with less training and inference time. Our model consistently performs better when organizing the label sequences from children to parents versus the inverse, as done in existing HTC approaches. Our experiments show that neither the label semantics nor an explicit graph encoder for the hierarchy is needed. This has strong practical implications for HTC as the architecture has fewer requirements and provides a speed-up by a factor of 2 at inference time. Moreover, training a separate decoder from scratch in conjunction with fine-tuning the encoder allows future researchers and practitioners to exchange the encoder part as new models arise. The source code is available at https://github.com/yousef-younes/RADAr.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。