对比零样本模型与树模型的硬件开销，发现前者效率极低但准确率不稳。
Light-Weight Benchmarks Reveal the Hidden Hardware Cost of Zero-Shot Tabular Foundation Models
- 构建可复现的基准测试，评估零样本模型在四大数据集上的精度与资源消耗
- 树模型在三数据集上准确率更高，推理耗时<0.4秒，内存占用<150MB且无需显存
- 零样本模型虽有小幅准确率提升，但显存需求高达9GB，延迟超960秒
零样本基础模型（FMs）承诺无需训练即可预测表格数据，但其硬件开销尚未被充分评估。本文提出一个完全可复现的基准测试，报告了在Adult-Income、Higgs-100k、Wine-Quality和California-Housing四个公开数据集上的测试准确率，以及单块NVIDIA T4 GPU上的运行时延、峰值CPU内存和峰值GPU显存。对比了两个开源基础模型TabPFN-1.0和TabICL-base与经过调优的XGBoost、LightGBM和随机森林基线。树集成模型在三个数据集上达到或超越FM的准确率，完整测试批处理耗时≤0.40秒，内存占用≤150MB，且无需显存。TabICL在Higgs数据集上提升0.8个百分点，但需约960秒延迟和9GB显存。TabPFN在Wine和Housing数据集上匹配树模型准确率，但峰值显存达4GB，无法处理10万行的Higgs全表。这些结果量化了当前表格基础模型在准确率与硬件开销之间的显著权衡，并为未来高效研究提供了开放基线。
原文摘要 · Abstract (English)
Zero-shot foundation models (FMs) promise training-free prediction on tabular data, yet their hardware footprint remains poorly characterized. We present a fully reproducible benchmark that reports test accuracy together with wall-clock latency, peak CPU RAM, and peak GPU VRAM on four public datasets: Adult-Income, Higgs-100k, Wine-Quality, and California-Housing. Two open FMs (TabPFN-1.0 and TabICL-base) are compared against tuned XGBoost, LightGBM, and Random Forest baselines on a single NVIDIA T4 GPU. The tree ensembles equal or surpass FM accuracy on three datasets while completing full-test batches in <= 0.40 s and <= 150 MB RAM, using zero VRAM. TabICL achieves a 0.8 percentage-point gain on Higgs but requires roughly 40,000 times more latency (960 s) and 9 GB VRAM. TabPFN matches tree-model accuracy on Wine and Housing but peaks at 4 GB VRAM and cannot process the full 100k-row Higgs table. These results quantify the substantial hardware-versus-accuracy trade-offs in current tabular FMs and provide an open baseline for future efficiency-oriented research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。