轻量版表格基础模型，一分钟完成预训练，适合教学与实验。
nanoTabPFN: A Lightweight and Educational Reimplementation of TabPFN
- 精简代码至万行以下，用预生成数据加速训练
- 单卡1分钟内达传统模型性能，快16万倍
- 专为教学和快速实验设计，易懂易用
表格基础模型如TabPFN已革新表格数据的预测机器学习。然而，其技术原理难以理解：现有开源实现代码超过一万行，流程复杂，缺乏架构文档与代码质量保障，难以入门，也不易用于新实验。我们提出nanoTabPFN，一种简化轻量的TabPFN v2架构实现及配套训练流程，使用预生成训练数据。在小数据设置下，仅需单卡1分钟预训练即可达到传统机器学习基线性能，较TabPFN v2快16万倍。该设计无需大型计算资源，使表格基础模型预训练更适用于教学场景。代码已公开于https://github.com/automl/nanoTabPFN。
原文摘要 · Abstract (English)
Tabular foundation models such as TabPFN have revolutionized predictive machine learning for tabular data. At the same time, the driving factors of this revolution are hard to understand. Existing open-source tabular foundation models are implemented in complicated pipelines boasting over 10,000 lines of code, lack architecture documentation or code quality. In short, the implementations are hard to understand, not beginner-friendly, and complicated to adapt for new experiments. We introduce nanoTabPFN, a simplified and lightweight implementation of the TabPFN v2 architecture and a corresponding training loop that uses pre-generated training data. nanoTabPFN makes tabular foundation models more accessible to students and researchers alike. For example, restricted to a small data setting it achieves a performance comparable to traditional machine learning baselines within one minute of pre-training on a single GPU (160,000x faster than TabPFN v2 pretraining). This eliminated requirement of large computational resources makes pre-training tabular foundation models accessible for educational purposes. Our code is available at https://github.com/automl/nanoTabPFN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。