仅用合成数据训练的表格大模型,性能超越多个工业级模型。
Mitra-v2 Technical Report

- 基于小规模2D Transformer,通过更广更大的合成数据预训练提升泛化能力。
- 参数量仅7700万却达到16亿参数模型水平,在300+真实数据集上表现顶尖。
- 开源可复现,适合需要高效、低成本表格预测的科研与工业场景。
我们提出Mitra-v2,一个仅在合成数据上预训练的表格基础模型,在真实世界分类与回归任务中表现卓越,涵盖信用风险评分、临床预测、设备故障检测和房价估算等。相比Mitra-v1,其预训练分布更大更丰富。基于小型2D Transformer架构,支持更长上下文与更大特征空间,优化改进使其能从更广泛的任务分布中学习。我们在包含300多个真实数据集的TabArena和TALENT基准上评估该模型,采用两种评估协议。在完整的TabArena基准上,Mitra-v2性能达到行业级模型如TabFM和EXAONE的水平,显著超越TabPFN-3。其仅7700万参数即媲美16亿参数的TabFM,成本仅为后者的5%。在TALENT上,它仍位居前列,明显优于TabPFN-3与TabICLv2。尤其在超过十类的分类任务中排名第一,尽管预训练时仅接触过最多十类的任务。这些结果使Mitra-v2成为目前最强且应用最广泛的开源表格基础模型之一。我们已将模型权重、推理与微调代码及评估结果以Apache-2.0许可开源。
原文摘要 · Abstract (English)
We introduce Mitra-v2, a tabular foundation model that delivers state-of-the-art performance on real-world classification and regression problems, from credit-risk scoring and clinical prediction to equipment-failure detection and house-price estimation. Mitra-v2 is trained only on synthetic data, with a pretraining distribution that is much larger and more diverse than Mitra-v1's. Built on a small 2D Transformer backbone, Mitra-v2 supports longer contexts and larger feature spaces. Improved optimization lets it learn from this larger task distribution. We evaluate Mitra-v2 on the TabArena and TALENT benchmarks, comprising more than 300 real-world datasets under two evaluation protocols. On the full TabArena benchmark, Mitra-v2 delivers state-of-the-art performance at the level of the industry-scale TabFM and EXAONE Tabular models, while surpassing TabPFN-3 by a wide margin in both classification and regression. Mitra-v2 matches the 1.6B-parameter TabFM with only 5% of its size (77M parameters), delivering frontier performance at a fraction of the cost. On TALENT, Mitra-v2 remains among the leading models, clearly outperforming TabPFN-3 and TabICLv2. It also ranks first on classification tasks with more than ten classes, even though it was pretrained only on tasks with at most ten classes. These results make Mitra-v2 one of the strongest and most broadly applicable open tabular foundation models released to date. We release the model weights, the inference and fine-tuning code, and our evaluation results under the Apache-2.0 license.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。