通过双流注意力机制，让表格模型同时学习原始特征与标签相关特征，自动过滤噪声。
SG-XDEAT: Sparsity-Guided Cross-Dimensional and Cross-Encoding Attention with Target-Aware Conditioning in Tabular Learning
- 双流编码：分别处理原始特征和标签相关的特征表示。
- 多维注意力交互：跨维度、跨编码层实现特征间深层关联建模。
- 自适应稀疏机制：动态抑制低效特征，提升模型对噪声的鲁棒性。
我们提出SG-XDEAT（稀疏引导的跨维度与跨编码注意力，带目标感知条件），一种面向表格数据监督学习的新框架。其核心是双流编码器，将每个输入特征分解为两个并行表示：原始值流和目标感知（标签感知）流。这两个流通过分层堆叠的注意力模块传播。SG-XDEAT集成三个关键组件：(i) 跨维度自注意力，捕捉每一流内部特征间的依赖关系；(ii) 跨编码自注意力，实现原始表示与目标感知表示之间的双向交互；(iii) 自适应稀疏自注意力（ASSA）机制，通过将低效标记的注意力权重动态趋近于零，有效抑制噪声影响。在多个公开基准上的实证结果表明，相比强基线模型，该方法持续取得性能提升，验证了联合建模原始与目标感知视图，并自适应过滤噪声，可构建更鲁棒的深度表格学习模型。
原文摘要 · Abstract (English)
We propose SG-XDEAT (Sparsity-Guided Cross Dimensional and Cross-Encoding Attention with Target Aware Conditioning), a novel framework designed for supervised learning on tabular data. At its core, SG-XDEAT employs a dual-stream encoder that decomposes each input feature into two parallel representations: a raw value stream and a target-conditioned (label-aware) stream. These dual representations are then propagated through a hierarchical stack of attention-based modules. SG-XDEAT integrates three key components: (i) Cross-Dimensional self-attention, which captures intra-view dependencies among features within each stream; (ii) Cross-Encoding self-attention, which enables bidirectional interaction between raw and target-aware representations; and (iii) an Adaptive Sparse Self-Attention (ASSA) mechanism, which dynamically suppresses low-utility tokens by driving their attention weights toward zero--thereby mitigating the impact of noise. Empirical results on multiple public benchmarks show consistent gains over strong baselines, confirming that jointly modeling raw and target-aware views--while adaptively filtering noise--yields a more robust deep tabular learner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。