树形线性模型可大幅压缩存储空间，适合海量标签文本分类。
Exploring space efficiency in a tree-based linear model for extreme multi-label classification
- 仅存储权重向量中的非零元素，利用稀疏性节省空间。
- 实验显示相比传统方法，存储空间最高可减少95%。
- 训练前可预估模型大小，避免不必要的剪枝操作。
极端多标签分类（XMC）旨在从大量标签中识别相关子集。树形线性模型因其高效和简洁而表现出色，但其空间复杂度未被充分研究。以往工作常因担心存储成本而采用剪枝等技术，可能导致性能下降。本文在稀疏数据假设下（文本数据常见），对树形模型的存储空间进行了理论与实证分析。发现树结构中训练二分类器时部分特征可能未被使用，导致权重向量出现零值。因此，仅存储非零元素可显著节省空间。实验表明，该方法在多标签文本分类任务中，相较标准一对多（one-vs-rest）方法，存储空间最多可减少95%。本研究提出一种简单的模型尺寸预估流程，可在训练前判断模型大小是否可接受，若已达标则无需进行权重剪枝等修改。
原文摘要 · Abstract (English)
Extreme multi-label classification (XMC) aims to identify relevant subsets from numerous labels. Among the various approaches for XMC, tree-based linear models are effective due to their superior efficiency and simplicity. However, the space complexity of tree-based methods is not well-studied. Many past works assume that storing the model is not affordable and apply techniques such as pruning to save space, which may lead to performance loss. In this work, we conduct both theoretical and empirical analyses on the space to store a tree model under the assumption of sparse data, a condition frequently met in text data. We found that, some features may be unused when training binary classifiers in a tree method, resulting in zero values in the weight vectors. Hence, storing only non-zero elements can greatly save space. Our experimental results indicate that tree models can achieve up to a 95% reduction in storage space compared to the standard one-vs-rest method for multi-label text classification. Our research provides a simple procedure to estimate the size of a tree model before training any classifier in the tree nodes. Then, if the model size is already acceptable, this approach can help avoid modifying the model through weight pruning or other techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。