轻量级输入空间适配器,无需改模型就能提升表格大模型性能。
TFM-Retouche: A Lightweight Input-Space Adapter for Tabular Foundation Models

- 在输入端加小修正,不改模型结构即可适配新数据。
- 在51个数据集上比原模型平均提升56分,训练和推理更快。
- 适合不想调模型又想提效的科研与工业用户。
表格基础模型(TFMs)如TabPFN-2.6、TabICLv2、ConTextTab、Mitra、LimiX和TabDPT,通过上下文学习实现强零样本性能,但其归纳偏置在推理时固定不变。将预训练的TFM适配到特定数据集或任务通常需要全量微调(计算开销大),或参数高效微调方法(如LoRA),但后者需针对每种模型架构定制。此外,权重空间微调是否提升准确率或校准性尚无定论。我们提出TFM-Retouche,一种轻量级、输入空间的残差适配器,对冻结的TFM骨干网络具有架构无关性。该适配器在输入空间学习一个小型残差修正,使输入数据与预训练模型的归纳偏置对齐。适配器通过冻结的TFM端到端训练,并配备后训练身份保护机制,在适应效果不佳时自动回退至未修改的TFM。在涵盖二分类、多分类和回归任务的51个数据集上的TabArena-Lite测试中,基于TabICLv2的TabICLv2-Retouche在排行榜上排名第一,采用轻量级每任务微调与集成策略,相比原始冻结的TabICLv2,聚合Elo得分提升+56,同时位于预测质量与训练/推理时间的帕累托前沿。
原文摘要 · Abstract (English)
Tabular foundation models (TFMs), such as TabPFN-2.6, TabICLv2, ConTextTab, Mitra, LimiX, and TabDPT, achieve strong zero-shot performance through in-context learning, but their inductive biases remain fixed at inference time. Adapting a pretrained TFM to a specific dataset or task typically requires either full fine-tuning, which is computationally expensive, or parameter-efficient tuning methods (PEFT) such as LoRA, which must be tailored to the internal architecture of each TFM. Furthermore, the evidence on whether weight-space fine-tuning improves accuracy or calibration is mixed \citep{tanna_exploring_2026,rubachev_finetuning_2025}. We introduce TFM-Retouche, a lightweight input-space residual adapter that is architecture-agnostic by design with respect to the frozen TFM backbone. TFM-Retouche learns a small residual correction in the input space to align the input data with the inductive biases of the pretrained model. The adapter is trained end-to-end through the frozen TFM, with a post-training identity guard that falls back to the unmodified TFM whenever adaptation does not help on held-out validation. On TabArena-Lite (51 datasets spanning binary classification, multiclass classification, and regression), TabICLv2-Retouche -- the framework instantiated on TabICLv2 -- is the top-ranked method on the leaderboard with light per-task tuning and ensembling, lifting aggregate Elo by +56 over the frozen TabICLv2 base and sitting on the Pareto front of predictive quality versus both training and inference time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。