用因果模型生成合成数据，提升小样本表格模型微调稳定性。
Causal Data Augmentation for Robust Fine-Tuning of Tabular Foundation Models
- 基于目标数据拟合因果模型，生成保持特征依赖的合成样本。
- 在33个数据集上将ROC-AUC中位数从0.10提升至0.12。
- 降低验证与测试性能相关性，让早停更可靠，适合小样本场景。
在数据稀缺下微调表格基础模型（TFMs）极具挑战，因验证数据更少时早停难以捕捉真实泛化性能。我们提出CausalMixFT，通过在目标数据上拟合结构因果模型（SCMs），生成结构一致的合成样本，增强微调鲁棒性与下游性能。该方法在TabArena的33个分类数据集及超过2300次微调运行中评估，将中位数标准化ROC-AUC从标准微调的0.10提升至0.12，优于纯统计生成器如CTGAN（-0.01）、TabEBM（-0.04）和TableAugment（-0.09）。同时，验证-测试性能相关性中位数由0.67降至0.30，使基于验证的早停更可靠，是提升数据稀缺下微调稳定性的关键步骤。结果表明，将因果结构融入数据增强，为低数据环境下微调表格基础模型提供了有效且原理清晰的路径。
原文摘要 · Abstract (English)
Fine-tuning tabular foundation models (TFMs) under data scarcity is challenging, as early stopping on even scarcer validation data often fails to capture true generalization performance. We propose CausalMixFT, a method that enhances fine-tuning robustness and downstream performance by generating structurally consistent synthetic samples using Structural Causal Models (SCMs) fitted on the target dataset. This approach augments limited real data with causally informed synthetic examples, preserving feature dependencies while expanding training diversity. Evaluated across 33 classification datasets from TabArena and over 2300 fine-tuning runs, our CausalMixFT method consistently improves median normalized ROC-AUC from 0.10 (standard fine-tuning) to 0.12, outperforming purely statistical generators such as CTGAN (-0.01), TabEBM (-0.04), and TableAugment (-0.09). Moreover, it narrows the median validation-test performance correlation gap from 0.67 to 0.30, enabling more reliable validation-based early stopping, a key step toward improving fine-tuning stability under data scarcity. These results demonstrate that incorporating causal structure into data augmentation provides an effective and principled route to fine-tuning tabular foundation models in low-data regimes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。