融合稀疏与稠密检索,提升长尾标签分类效果
Ranking-based Fusion Algorithms for Extreme Multi-label Text Classification (XMTC)
- 基于排名融合策略,结合BM25与BERT等多方法预测结果
- 在头尾标签上均实现更好性能,尤其改善低频标签识别
- 适合需要兼顾高频率与稀有标签的文本分类任务
在极端多标签文本分类(XMTC)中,标签来自大规模标签空间,且标签分布呈长尾特征:高频高覆盖的‘头标签’与低频低覆盖的‘尾标签’并存,导致模型难以均衡表现。为应对这一挑战,本文提出融合多种检索方法的方案,结合基于词袋的稀疏检索器(如BM25)与基于语义嵌入的稠密检索器(如微调BERT)。稀疏检索器擅长精确匹配关键词,稠密检索器则通过共享嵌入空间中的近似最近邻算法捕捉语义相似性。通过基于排名的融合机制,将两者互补优势整合,生成更优的最终排序,显著提升头标签与尾标签的整体分类效果。
原文摘要 · Abstract (English)
In the context of Extreme Multi-label Text Classification (XMTC), where labels are assigned to text instances from a large label space, the long-tail distribution of labels presents a significant challenge. Labels can be broadly categorized into frequent, high-coverage \textbf{head labels} and infrequent, low-coverage \textbf{tail labels}, complicating the task of balancing effectiveness across all labels. To address this, combining predictions from multiple retrieval methods, such as sparse retrievers (e.g., BM25) and dense retrievers (e.g., fine-tuned BERT), offers a promising solution. The fusion of \textit{sparse} and \textit{dense} retrievers is motivated by the complementary ranking characteristics of these methods. Sparse retrievers compute relevance scores based on high-dimensional, bag-of-words representations, while dense retrievers utilize approximate nearest neighbor (ANN) algorithms on dense text and label embeddings within a shared embedding space. Rank-based fusion algorithms leverage these differences by combining the precise matching capabilities of sparse retrievers with the semantic richness of dense retrievers, thereby producing a final ranking that improves the effectiveness across both head and tail labels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。