arXiv:2511.09970cs.LGcs.AI2025-11中稿 · AAAI

首个专为大规模表格数据设计的多任务Transformer,提升多任务学习效果。

MultiTab: A Scalable Foundation for Multitask Learning on Tabular Data

  • 提出多任务掩码注意力机制,动态建模特征间依赖关系。
  • 在推荐、社会经济、物理等多类数据上均优于现有模型,任务越多越显著。
  • 适合需要处理多目标表格数据的研究者和工业应用开发者。

表格数据是全球最丰富的数据类型,广泛应用于金融、医疗、电商等领域。随着表格数据规模扩大并涵盖多个相关目标,利用共享任务信息以提升多任务泛化能力的需求日益迫切。多任务学习(MTL)已成为提升泛化与效率的有效手段,但现有研究主要聚焦于大规模推荐系统,其在更广泛表格领域中的潜力尚未充分挖掘。此外,现有表格数据的MTL方法大多基于多层感知机架构,难以捕捉复杂特征交互,且在数据丰富时扩展性差,而变换器架构已在其他领域解决了这一问题。为此,我们提出MultiTab-Net,首个专为大规模表格数据设计的多任务变换器架构。MultiTab-Net采用新颖的多任务掩码注意力机制,动态建模特征-特征依赖关系,同时缓解任务竞争。大量实验表明,MultiTab-Net在包括大规模推荐数据、人口统计学数据及物理数据在内的多种领域中,持续优于现有MTL架构和单任务变换器,覆盖不同任务数量、任务类型与特征模态。此外,我们还构建了MultiTab-Bench——一个可系统评估多任务动态性的广义合成数据生成器,支持调节任务数量、任务相关性和相对任务复杂度。代码已公开于https://github.com/Armanfard-Lab/MultiTab。

原文摘要 · Abstract (English)

Tabular data is the most abundant data type in the world, powering systems in finance, healthcare, e-commerce, and beyond. As tabular datasets grow and span multiple related targets, there is an increasing need to exploit shared task information for improved multitask generalization. Multitask learning (MTL) has emerged as a powerful way to improve generalization and efficiency, yet most existing work focuses narrowly on large-scale recommendation systems, leaving its potential in broader tabular domains largely underexplored. Also, existing MTL approaches for tabular data predominantly rely on multi-layer perceptron-based backbones, which struggle to capture complex feature interactions and often fail to scale when data is abundant, a limitation that transformer architectures have overcome in other domains. Motivated by this, we introduce MultiTab-Net, the first multitask transformer architecture specifically designed for large tabular data. MultiTab-Net employs a novel multitask masked-attention mechanism that dynamically models feature-feature dependencies while mitigating task competition. Through extensive experiments, we show that MultiTab-Net consistently achieves higher multitask gain than existing MTL architectures and single-task transformers across diverse domains including large-scale recommendation data, census-like socioeconomic data, and physics datasets, spanning a wide range of task counts, task types, and feature modalities. In addition, we contribute MultiTab-Bench, a generalized multitask synthetic dataset generator that enables systematic evaluation of multitask dynamics by tuning task count, task correlations, and relative task complexity. Our code is publicly available at https://github.com/Armanfard-Lab/MultiTab.

多任务学习表格数据Transformer机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。