LassoFlexNet让深度模型在表格数据上表现超越树模型，还能像Lasso一样解释。
LassoFlexNet: Flexible Neural Architecture for Tabular Data
- 用每特征嵌入和组Lasso稀疏选择关键变量，融合五种先验知识
- 52个数据集上最高比树模型提升10%，且保持可解释性
- 专为深层表数据设计，适合需要高精度与可解释性的场景
尽管深度神经网络在视觉和语言领域占据主导地位，但在表格数据上常落后于树模型。为弥合这一差距，我们引入五种关键归纳偏置：对无关特征的鲁棒性、轴对齐性、局部不规则性、特征异质性及训练稳定性。提出LassoFlexNet架构，通过每特征嵌入评估输入的线性和非线性边际贡献，并利用绑定组Lasso机制稀疏选择相关变量。由于这些组件带来优化挑战，使标准近端方法不稳定，我们设计了带指数移动平均（EMA）的分层序贯近端自适应梯度优化器，确保稳定收敛。在三个基准的52个数据集上，LassoFlexNet达到或超过领先的树模型性能，相对提升高达10%，同时保持类似Lasso的可解释性。消融实验和理论证明验证了其增强的表达能力及打破不良旋转不变性的结构优势。
原文摘要 · Abstract (English)
Despite their dominance in vision and language, deep neural networks often underperform relative to tree-based models on tabular data. To bridge this gap, we incorporate five key inductive biases into deep learning: robustness to irrelevant features, axis alignment, localized irregularities, feature heterogeneity, and training stability. We propose \emph{LassoFlexNet}, an architecture that evaluates the linear and nonlinear marginal contribution of each input via Per-Feature Embeddings, and sparsely selects relevant variables using a Tied Group Lasso mechanism. Because these components introduce optimization challenges that destabilize standard proximal methods, we develop a \emph{Sequential Hierarchical Proximal Adaptive Gradient optimizer with exponential moving averages (EMA)} to ensure stable convergence. Across $52$ datasets from three benchmarks, LassoFlexNet matches or outperforms leading tree-based models, achieving up to a $10$\% relative gain, while maintaining Lasso-like interpretability. We substantiate these empirical results with ablation studies and theoretical proofs confirming the architecture's enhanced expressivity and structural breaking of undesired rotational invariance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。