arXiv:2605.18971cs.LGcs.AI2026-05

用更真实的合成数据提升表格模型性能和鲁棒性

Shaping the Prior: How Synthetic Task Distributions Determine Tabular Foundation Model Quality

论文配图:Shaping the Prior: How Synthetic Task Distributions Determine Tabular Foundation Model Quality
图 1 · 摘自论文原文
  • 构建四组件合成先验O'Prior,模拟真实数据的复杂特性
  • 在多个真实表格数据集上显著提升准确率与抗分布偏移能力
  • 适合关注表格数据建模鲁棒性的研究者与工程师

表格基础模型的质量由何决定?与语言或视觉不同,表格模型的归纳偏置几乎完全来自合成预训练分布,但这类分布的设计仍不清晰。标准合成先验过于理想化,忽略了影响部署鲁棒性的不规则性和失效模式。本文提出O'Prior,一种基于四个耦合组件的组合式现实先验:跨越多种功能族的分层因果模型元生成器;涵盖异质边缘分布、缺失值和目标变换的模块化现实引擎;显式的压力模块,注入混淆因素与支持-查询不匹配;以及受课程引导、防泄露的生成协议。为隔离先验设计的影响,固定架构、优化器和计算预算,仅改变合成任务分布。O'Prior在多个真实表格基准上持续显著提升下游准确率与鲁棒性,增益集中于分布不规则的场景。消融实验证实机制多样性、现实组合性及扰动感知压力各自独立贡献,效果不可互换。结果表明,合成先验构造是表格基础模型质量的一阶且被严重忽视的关键因素。

原文摘要 · Abstract (English)

What determines the quality of a tabular foundation model? Unlike language or vision, tabular foundation models acquire their inductive biases almost entirely from synthetic pretraining distributions, yet the design of these distributions remains poorly understood. Standard synthetic priors are too well-behaved: they omit the irregularities and failure modes that determine deployment robustness. We introduce O'Prior, a compositional realism prior built around four coupled components: a hierarchical SCM meta-generator spanning diverse functional families; a modular realism engine covering heterogeneous marginals, missingness, and target transforms; an explicit stress module injecting confounding and support-query mismatch; and a curriculum-governed, leakage-safe generation protocol. To isolate prior design as the scientific variable, we hold architecture, optimizer, and compute budget fixed and vary only the synthetic task distribution. O'Prior yields consistent and substantial improvements in downstream accuracy and robustness across real tabular benchmarks, with gains concentrated in regimes characterized by distributional irregularities. Ablations confirm that mechanism diversity, realism composition, and shift-aware stress each contribute independently, their effects are not interchangeable. These results establish synthetic prior construction as a first-order and largely overlooked determinant of tabular foundation model quality

表格模型合成数据先验设计鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。