让表格模型一次预测多个目标,效率提升显著
TabPFN-MT: A Natively Multitask In-Context Learner for Tabular Data

- 用共享解码头和扩展标签编码器实现多任务上下文学习
- 在344个数据集上达到新纪录,多任务平均排名4.89
- 推理成本从O(T)降至O(1),适合小样本多目标场景
先前的基于先验数据拟合网络(PFNs)在表格数据中表现优异,但仅支持单任务推理,需多次前向传播且无法共享任务间信息。本文提出TabPFN-MT,通过在多目标合成先验数据上训练,引入扩展的y-编码器与共享解码头,实现多任务上下文学习与并行预测。该模型依赖上下文学习而非传统梯度训练,专为小到中等规模数据集设计。在平均样本数少于1,000的数据集上,对344个数据集的评估表明,其在深度表格多任务学习中达到新最优性能。尽管存在联合优化的计算不对称性,其性能仍优于最新单任务集成模型。尤其在多任务数据集上,整体准确率平均排名达4.89,为所有测试模型中最高。关键优势在于,将T个任务的推理成本从O(T)降低至O(1),极大提升多目标表格应用的计算效率。
原文摘要 · Abstract (English)
Prior-Data Fitted networks (PFNs) have been very successful in tabular contexts, handling prediction tasks in context. However, they are designed for single-task inference, meaning that predicting several target values within a context requires repeated forward calls and precludes inter-task information sharing. We propose TabPFN-MT, which is trained on an expanded multi-target synthetic prior to capture inter-task dependencies in context. This model uses an expanded $y$-encoder and a shared decoder head to enable multitask in-context learning and simultaneous inference. The model is uniquely specialized for small-to-medium datasets by relying on in-context learning rather than traditional gradient-based training. Within this regime (averaging fewer than 1,000 samples), extensive evaluations across 344 datasets demonstrate that TabPFN-MT establishes a new state-of-the-art for deep tabular multitask learning. Furthermore, despite the inherent compute asymmetry of joint optimization, our model remains highly competitive with the latest state-of-the-art single-task ensembles. Notably, on multitask datasets it achieves an overall Accuracy rank of 4.89, the highest average rank among all models tested. Crucially, TabPFN-MT delivers this highly competitive performance while reducing the inference cost for $T$ tasks from $O(T)$ to $O(1)$ forward passes, offering a massive computational efficiency improvement for multi-target tabular applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。