针对表格数据异质性，提出多分支注意力框架提升分类回归精度。
Multi-branch of Attention Yields Accurate Results for Tabular Data
- 设计多分支注意力机制并行处理异质特征，融合效果好且参数增长小。
- 在多个数据集上超越现有基于Transformer的方法，分类与回归均表现更优。
- 适合需要高精度表格建模的场景，尤其对特征差异大的数据有效。
表格数据天然具有显著的特征异质性，但现有基于Transformer的方法缺乏针对性处理机制。为此，我们提出MAYA——一种编码器-解码器结构的Transformer框架。编码器中设计了多分支注意力（MBA），通过构建多个并行注意力分支并平均各分支特征，有效融合异质特征的同时控制参数增长；同时引入带动态一致性约束的协同学习，生成更鲁棒的表征。解码阶段采用交叉注意力，无缝整合表格数据与对应标签特征，双重注意力机制能有效捕捉实例内与实例间交互。我们在多个数据集上评估该方法，并与多种先进Transformer基线对比。大量实验表明，该模型在表格分类与回归任务中均优于现有基于Transformer的方法。
原文摘要 · Abstract (English)
Tabular data inherently exhibits significant feature heterogeneity, but existing transformer-based methods lack specialized mechanisms to handle this property. To bridge the gap, we propose MAYA, an encoder-decoder transformer-based framework. In the encoder, we design a Multi-Branch of Attention (MBA) that constructs multiple parallel attention branches and averages the features at each branch, effectively fusing heterogeneous features while limiting parameter growth. Additionally, we employ collaborative learning with a dynamic consistency weight constraint to produce more robust representations. In the decoder stage, cross-attention is utilized to seamlessly integrate tabular data with corresponding label features. This dual-attention mechanism effectively captures both intra-instance and inter-instance interactions. We evaluate the proposed method on a wide range of datasets and compare it with other state-of-the-art transformer-based methods. Extensive experiments demonstrate that our model achieves superior performance among transformer-based methods in both tabular classification and regression tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。