EXAONE Tabular用新架构实现高效表格推理,零微调即达顶尖性能。
EXAONE Tabular 1.0 : Technical Report

- 在Transformer中交错设计特征与样本注意力机制,用摘要标记协同控制。
- 2081万参数模型在TabArena分类榜首,推理成本仅为大模型的1/11。
- 适合追求高效率、无需微调的表格建模任务,尤其适合工业部署。
EXAONE Tabular 是一个紧凑的表格基础模型家族,通过上下文学习完成分类与回归任务,无需针对特定数据集进行梯度更新。其核心贡献在于对表格上下文学习的架构级重构:不再将特征压缩为固定行嵌入后由独立行级学习器处理,而是将特征轴注意力与支持条件下的项轴注意力在每一层Transformer中交错整合,由项摘要和特征摘要标记协调。在四个公开基准上,该模型兼具强预测性能与高效率。在TabArena上,20.81M参数分类模型整体排名第一,超越调优集成模型与4小时AutoML流水线;回归任务达到16.4亿参数的TabFM性能水平,推理成本仅约其1/11。在BCCO与TALENT上,分类排名第二,回归排名第一。在ScoringBench上,点估计与预测分布质量均获最佳平均排名,$R^2$、RMSE与CRPS三项指标领先。综合表现确立EXAONE Tabular为当前最高效的表格基础模型家族,覆盖分类、点回归与概率回归任务。
原文摘要 · Abstract (English)
EXAONE Tabular is a compact tabular foundation model family for classification and regression via in-context learning, producing predictions without dataset-specific gradient updates. Pretrained exclusively on a synthetic structural-causal-model (SCM) prior, its central contribution is an architecture-centered redesign of tabular in-context learning. Rather than compressing features into a fixed row embedding before a separate row-level learner, EXAONE Tabular interleaves feature-axis attention within each item with support-conditioned item-axis attention within each feature at every Transformer layer, mediated by item-summary and feature-summary tokens. Across four public benchmarks, EXAONE Tabular combines strong predictive performance with high efficiency. On TabArena, its 20.81M-parameter classification model ranks first overall, surpassing tuned ensembles and 4-hour AutoML pipelines, while regression reaches the performance regime of the 1.64B-parameter TabFM at roughly 1/11 the inference cost. On BCCO and TALENT, EXAONE Tabular ranks second in classification and first in regression. On ScoringBench, it achieves the best mean rank for both point-estimation and predictive-distribution quality, leading the $R^2$, RMSE, and CRPS evaluations. Together, these results establish EXAONE Tabular as a state-of-the-art compact tabular foundation model family, combining strong predictive performance across classification, point regression, and probabilistic regression with an efficient model design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。