arXiv:2602.02523cs.LGcs.AI2026-02被引 1

测试表格模型在确定性计算外推中的能力,发现传统模型擅长平滑拟合,但难保精确结果。

TabularMath: Evaluating Computational Extrapolation in Tabular Learning via Program-Verified Synthesis

  • 基于验证程序生成114个确定性问题,构建可验证的表格外推基准
  • TabPFN在分布内表现极佳(R²=0.998),但外推时精确匹配率低于10%
  • 大模型上下文学习(ICL)在少样本下保持约40%精确匹配,适合高精度计算场景

标准表格基准主要评估模型在数据流形内的插值能力,而大量高价值表格数据(如金融建模、物理仿真)由确定性计算过程生成。本文提出TabularMath,一个包含114个确定性问题(233,472行)的诊断基准,基于GSM8K和AIME的验证程序合成。评估了9种表格架构与基于GPT-OSS-120B的上下文学习(ICL)。在标准回归指标上,TabPFN v2.5表现优异,分布内R²=0.998,且在分布偏移下仍保持正R²,是所测模型中唯一具备此特性的。然而,在四舍五入一致性(精确整数匹配)指标下,其外推性能降至10%以下,而ICL维持约40%。该差距表明,表格模型擅长光滑函数逼近,但在外推时难以恢复精确计算输出。两种范式互补:TabPFN高效扩展数据;ICL以少量示例实现精确计算。代码与数据已开源。

原文摘要 · Abstract (English)

Standard tabular benchmarks mainly focus on the evaluation of a model's capability to interpolate values inside a data manifold, where models good at performing local statistical smoothing are rewarded. However, there exists a very large category of high-value tabular data, including financial modeling and physical simulations, which are generated based upon deterministic computational processes, as opposed to stochastic and noisy relationships. Therefore, we investigate if tabular models can provide an extension from statistical interpolation to computational extrapolation. We propose TabularMath, a diagnostic benchmark of 114 deterministic problems (233,472 rows) generated from verified programs based on GSM8K and AIME. We evaluate 9 tabular architectures and in-context learning (ICL) with GPT-OSS-120B. On standard regression metrics, TabPFN v2.5 performs remarkably well, achieving R^2=0.998 in-distribution and maintaining positive R^2 even under distribution shift, which is unique among the tabular models we tested. When we measure rounded consistency (exact integer match), a different picture emerges: TabPFN v2.5 drops below 10% on out-of-distribution data, while ICL maintains around 40%. This gap between R^2 and exact-match accuracy suggests that tabular models learn smooth function approximations but struggle to recover precise computational outputs under extrapolation. The two paradigms appear complementary: TabPFN scales efficiently with data; ICL achieves exact computation from few examples. We release all code and data to support further investigation.

表格学习外推计算验证大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。