评测大模型在少样本表格分类中的表现，发现零样本有时比有样本还强。
LLMTabBench: Evaluating LLMs on Binary Tabular Classification From Zero to Few Shots
- 构建新基准LLMTabBench，测试大模型在低数据下的表格分类能力。
- 零样本下大模型表现常优于有少量标注样本的情况。
- 数据复杂度超阈值后性能下降，少量示例反而拖累效果。
表格数据上的监督分类是机器学习的核心任务，但其对大规模标注数据的依赖限制了在数据稀缺场景的应用。少样本方法如TabPFN通过大规模合成预训练实现优异性能，但仍需标注上下文样本。大语言模型（LLMs）可通过任务描述实现零样本和少样本上下文学习，但在表格数据上的表现仍不稳定。我们提出LLMTabBench，一个用于评估大模型在低数据条件下表格分类表现的基准。该基准研究大模型先验知识与任务描述、少量样本之间的交互关系，并分析真实世界与受控合成数据集上性能随数据复杂度的变化。结果表明，大模型在零样本设置下具有高度竞争力，有时甚至超越提供少量样本的模型。然而，额外样本可能与先验知识冲突，导致性能下降。我们还观察到存在一个复杂度阈值，超过后大模型性能下降，少样本示例变得不再有效。这些发现揭示了上下文学习在表格数据中的关键局限，为大模型在低数据场景的部署提供了指导。
原文摘要 · Abstract (English)
Supervised classification on tabular data remains a central machine learning task, but its dependence on large labeled datasets limits its applicability in data-scarce settings. Few-shot methods such as TabPFN achieve strong performance through large-scale synthetic pretraining, yet still require labeled context examples. Large Language Models (LLMs) offer a more flexible alternative through zero- and few-shot in-context learning from task descriptions, but their behavior on tabular data remains inconsistent. We introduce LLMTabBench, a benchmark for evaluating LLMs on tabular classification under low-data conditions. The benchmark studies how LLM prior knowledge interacts with task descriptions and few-shot examples, and how performance changes with increasing data complexity across real-world and controlled synthetic datasets. We find that LLMs can be highly competitive in zero-shot settings, sometimes outperforming models given few-shot examples. However, additional examples may conflict with prior knowledge, thereby degrading performance. We also observe a complexity threshold at which LLM performance declines and few-shot examples become less useful. These results clarify key limits of in-context learning for tabular data and inform the deployment of LLMs in low-data regimes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。