探索表格大模型微调效果，发现多数情况不提升性能。
Exploring Fine-Tuning for Tabular Foundation Models
- 对比零样本、元学习、全量与参数高效微调方法
- 微调常降低准确率或校准质量，受数据特性影响大
- 提供何时微调有效及适用场景的实用建议
表格基础模型（TFMs）在结构化数据上展现出强大的上下文学习能力，零样本性能已接近传统机器学习方法。本研究发现，零样本TFM已有较强表现，而微调收益高度依赖模型与数据。元学习和参数高效微调（PEFT）在特定条件下带来适度提升，但全监督微调（SFT）常导致准确率或校准质量下降。本文首次在TALENT、OpenML-CC18和TabZilla等基准上系统评估了各类微调策略，分析数据不平衡性、规模与维度对结果的影响，覆盖性能、校准与公平性，为微调是否有益提供实践指导。
原文摘要 · Abstract (English)
Tabular Foundation Models (TFMs) have recently shown strong in-context learning capabilities on structured data, achieving zero-shot performance comparable to traditional machine learning methods. We find that zero-shot TFMs already achieve strong performance, while the benefits of fine-tuning are highly model and data-dependent. Meta-learning and PEFT provide moderate gains under specific conditions, whereas full supervised fine-tuning (SFT) often reduces accuracy or calibration quality. This work presents the first comprehensive study of fine-tuning in TFMs across benchmarks including TALENT, OpenML-CC18, and TabZilla. We compare Zero-Shot, Meta-Learning, Supervised (SFT), and parameter-efficient (PEFT) approaches, analyzing how dataset factors such as imbalance, size, and dimensionality affect outcomes. Our findings cover performance, calibration, and fairness, offering practical guidelines on when fine-tuning is most beneficial and its limitations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。