轻量级表格基础模型TabSwift，推理更快更省资源。
TabSwift: An Efficient Tabular Foundation Model with Row-Wise Attention
- 用行级注意力+门控稳定机制，保持高性能
- 比现有模型快2倍以上，支持即时预测
- 适合实时部署的低延迟场景
表格基础模型如TabPFN通过上下文学习实现预测，直接从已标注训练样本推断测试标签，在小到中等规模数据集上表现优异。然而，当前模型为提升准确率不断增加架构复杂度，导致推理成本上升，限制实际应用。本文重新审视原始TabPFN设计，提出仅使用行级注意力的轻量级主干网络，并引入门控注意力稳定机制和少量可学习的注册令牌以提供全局上下文，显著提升预训练质量。由此得到的模型TabSwift在分类与回归任务上均表现良好，性能媲美更强的TabPFN v2和TabICL，同时推理效率更高。针对延迟敏感服务，进一步设计层间自适应早停机制，根据样本动态调整推理深度。整体上，TabSwift实现了高效且任意时间响应的表格上下文学习，适用于实际部署。
原文摘要 · Abstract (English)
Tabular foundation models, exemplified by TabPFN, perform prediction via in-context learning, inferring test labels directly from labeled training examples. They have demonstrated competitive performance, particularly on small-to-medium datasets. However, recent tabular foundation models often improve accuracy with increasingly complex architectures, incurring higher inference cost and limiting practical deployment. In this work, we revisit the original TabPFN design and show that a lightweight row-wise attention-only backbone can remain highly competitive with two simple enhancements: a gated attention stabilization mechanism and a small set of learnable register tokens that provide global context and improve pretraining quality. The resulting model, TabSwift, supports both classification and regression, and is competitive with stronger tabular foundation models (e.g., TabPFN v2 and TabICL) while being more efficient at inference. For latency-sensitive serving, we further introduce an adaptive layer-wise early-exit mechanism that dynamically adjusts inference depth per sample. Overall, TabSwift enables efficient and anytime tabular in-context learning for practical deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。