arXiv:2605.14365cs.LGcs.AI2026-05

提出可调控多样性的表格神经网络集成方法,提升模型稳定性与泛化能力。

LoMETab: Beyond Rank-1 Ensembles for Tabular Deep Learning

论文配图:LoMETab: Beyond Rank-1 Ensembles for Tabular Deep Learning
图 1 · 摘自论文原文
  • 通过低秩矩阵分解实现多成员权重的隐式集成,支持灵活控制多样性。
  • 在多个数据集上验证了预测多样性随参数配置变化达数个数量级。
  • 适用于追求模型稳定性和可解释性的表格学习任务研究者。

近期表格学习基准显示主流方法性能趋于紧密聚集,缺乏明显优劣层级,涵盖梯度提升树、注意力架构及隐式集成如TabM。随着基准性能趋稳,理解并控制简单神经表格模型的竞争机制成为关键。本文提出LoMETab,一种秩-$r$的乘法隐式集成推广形式。将原有的秩-1 BatchEnsemble/TabM调制扩展为秩-$r$的恒等残差哈达玛族,通过参数化每个成员权重为 $W_k = W igodot (1 + A_kB_k^ op)$,其中 $W$ 共享,$(A_k, B_k)$ 为成员特定的低秩因子。该设计揭示两个实用的多样性调控轴:适配器秩 $r$ 和初始化尺度 $σ_{\mathrm{init}}$,并证明当 $r \ge 2$ 时,该推广严格扩大了BatchEnsemble的假设类。实验表明,这种增强容量在训练后体现为可观测的预测多样性:在代表性分类数据集上,LoMETab的成对KL值显著高于加性低秩消融,且 $(r, σ_{\mathrm{init}})$ 可实现高达数个数量级的成对KL变化。这种多样性反映在任务相关的输出层面指标中:分类任务表现为类别预测分歧,回归任务表现为不确定性,说明控制效果超越成对KL,延伸至决策和输出层面。在 $(r, σ_{\mathrm{init}})$ 网格上的系统实验显示,性能表现依赖于数据集,支持LoMETab作为可控的隐式集成家族而非固定秩-1结构。

原文摘要 · Abstract (English)

Recent tabular learning benchmarks increasingly show a tight performance cluster rather than a clear hierarchy among leading methods, spanning gradient boosted decision trees, attention-based architectures, and implicit ensembles such as TabM. As benchmark gains plateau, a complementary goal is to understand and control the mechanisms that make simple neural tabular models competitive. We propose LoMETab, a rank-$r$ generalization of multiplicative implicit ensembles. LoMETab lifts the rank-1 BatchEnsemble/TabM modulation to a rank-$r$ identity-residual Hadamard family by parameterizing each member weight as $W_k = W \odot (1 + A_kB_k^\top)$, where $W$ is shared and $(A_k, B_k)$ are member-specific low-rank factors. This exposes two practical diversity-control axes: the adapter rank $r$ and the initialization scale $σ_{\mathrm{init}}$, and we prove that for $r \ge 2$ this generalization strictly enlarges BatchEnsemble's hypothesis class. Empirically, we show that this added capacity manifests as measurable predictive diversity after training: on representative classification datasets, LoMETab sustains higher pairwise KL than an additive low-rank ablation, and $(r, σ_{\mathrm{init}})$ provides broad control over pairwise KL, varying by up to several orders of magnitude across configurations. The induced diversity is reflected in task-appropriate output-level measures: argmax disagreement for classification and ambiguity for regression, indicating that the control extends beyond pairwise KL to decision- and output-level member variation. Finally, experiments sweeping over adapter rank $r$ and initialization scale $σ_{\mathrm{init}}$ reveal that predictive performance is dataset-dependent over the $(r, σ_{\mathrm{init}})$ grid, supporting LoMETab as a controllable family of implicit ensembles rather than a fixed rank-1 construction.

表格学习隐式集成多样性控制低秩建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。