用视觉增强大模型提升极端多标签分类性能,效果超越现有方法。
Large Language Models Meet Extreme Multi-label Classification: Scaling and Multi-modal Framework
- 用小规模解码器模型结合视觉信息,实现高效多模态学习。
- 在最大数据集上性能提升最高达8.21%(P@1),显著优于纯文本模型。
- 适合关注多模态、大规模标签分类的科研与工程人员使用。
基础模型在众多领域推动了人工智能的变革,但在极端多标签分类(XMC)中的潜力尚未充分挖掘。XMC需从超大规模标签空间中为查询匹配相关标签,关键在于平衡效率与性能。近期方法通常将XMC建模为小编码器仅架构学习的嵌入向量之间的最大内积搜索。本文解决两个核心问题:如何有效利用更大规模的解码器模型,以及如何在保持计算效率的前提下融合视觉信息。我们证明二者均对XMC至关重要,且可协同提升性能。结果显示,仅数亿参数的解码器模型即可带来显著改进,同时控制计算开销。我们提出的视觉增强型极端多标签学习框架(ViXML)通过每张图像提取单一嵌入来高效集成基础视觉模型,避免计算量激增,释放多模态能力。令人惊讶的是,采用小编码器的ViXML在多数情况下超越纯文本解码器模型,验证了‘一张图胜过数十亿参数’。此外,我们扩展了现有纯文本数据集,引入视觉元数据,供未来基准测试使用。在四个公开文本数据集及其对应的图像增强版本上的综合实验表明,我们的方法显著优于先前最优,最大数据集上P@1指标提升高达8.21%。代码已开源:https://github.com/DiegoOrtego/vixml。
原文摘要 · Abstract (English)
Foundation models have revolutionized artificial intelligence across numerous domains, yet their transformative potential remains largely untapped in Extreme Multi-label Classification (XMC). Queries in XMC are associated with relevant labels from extremely large label spaces, where it is critical to strike a balance between efficiency and performance. Therefore, many recent approaches efficiently pose XMC as a maximum inner product search between embeddings learned from small encoder-only transformer architectures. In this paper, we address two important aspects in XMC: how to effectively harness larger decoder-only models, and how to exploit visual information while maintaining computational efficiency. We demonstrate that both play a critical role in XMC separately and can be combined for improved performance. We show that a few billion-size decoder can deliver substantial improvements while keeping computational overhead manageable. Furthermore, our Vision-enhanced eXtreme Multi-label Learning framework (ViXML) efficiently integrates foundation vision models by pooling a single embedding per image. This limits computational growth while unlocking multi-modal capabilities. Remarkably, ViXML with small encoders outperforms text-only decoder in most cases, showing that an image is worth billions of parameters. Finally, we present an extension of existing text-only datasets to exploit visual metadata and make them available for future benchmarking. Comprehensive experiments across four public text-only datasets and their corresponding image enhanced versions validate our proposals' effectiveness, surpassing previous state-of-the-art by up to +8.21\% in P@1 on the largest dataset. ViXML's code is available at https://github.com/DiegoOrtego/vixml.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。