重新审视层次文本分类的评估方式，发现简单基线常胜过复杂模型。
Revisiting Hierarchical Text Classification: Inference and Metrics
- 用专门设计的层级指标替代传统多标签评估
- 新数据集上复杂模型多数不如简单基线表现好
- 强调评估方法对结果解读的关键影响
层次文本分类（HTC）是将文本分配到具有层级结构的标签空间中的任务。近期研究将其视为常规多标签分类问题进行评估。本文提出应采用专门设计的层级度量来评估模型，并揭示了度量选择与预测推理方法的复杂性。我们引入一个更具挑战性的新数据集，对近期先进模型进行了公平评估，同时与多种简单但强劲的基线模型对比，包括一种理论驱动的新损失函数。结果表明，这些基线在多数情况下均能与最新模型相媲美。这凸显了在提出新方法时需谨慎考虑评估方法的重要性。代码与数据集见 https://github.com/RomanPlaud/revisitingHTC。
原文摘要 · Abstract (English)
Hierarchical text classification (HTC) is the task of assigning labels to a text within a structured space organized as a hierarchy. Recent works treat HTC as a conventional multilabel classification problem, therefore evaluating it as such. We instead propose to evaluate models based on specifically designed hierarchical metrics and we demonstrate the intricacy of metric choice and prediction inference method. We introduce a new challenging dataset and we evaluate fairly, recent sophisticated models, comparing them with a range of simple but strong baselines, including a new theoretically motivated loss. Finally, we show that those baselines are very often competitive with the latest models. This highlights the importance of carefully considering the evaluation methodology when proposing new methods for HTC. Code implementation and dataset are available at \url{https://github.com/RomanPlaud/revisitingHTC}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。