用双轴注意力+元学习，让表格数据少样本学习更高效。
Orion-Bix: Bi-Axial Attention for Tabular In-Context Learning
- 双轴注意力融合局部与全局依赖，提升表格特征捕捉能力。
- 在公开基准上超越梯度提升基线，少样本性能接近顶尖模型。
- 适合需要快速适配新表格任务的工业场景，支持大标签空间。
表格数据驱动大多数现实世界机器学习应用，但构建通用模型仍具挑战：数值与类别字段混合、特征结构弱、标注数据有限。为此，我们提出Orion-Bix，一种结合双轴注意力与元学习的上下文学习表格基础模型，用于少样本表格学习。其编码器交替使用标准、分组、层次化和关系注意力，通过多-CLS汇总融合输出，高效捕捉局部与全局依赖。标签感知的ICL头可动态适应并借助分层决策路由扩展至大规模标签空间。模型在合成生成、结构多样且带因果先验的表格上进行元训练，学习跨异构数据的可迁移归纳偏置。作为兼容scikit-learn的基础模型，Orion-Bix在公共基准上优于梯度提升基线，且与当前最先进表格基础模型性能相当，表明双轴注意力结合情景式元训练能实现稳健的少样本表格学习。代码已开源：https://github.com/Lexsi-Labs/Orion-BiX。
原文摘要 · Abstract (English)
Tabular data drive most real-world machine learning applications, yet building general-purpose models for them remains difficult. Mixed numeric and categorical fields, weak feature structure, and limited labeled data make scaling and generalization challenging. To this end, we introduce Orion-Bix, a tabular foundation model that combines biaxial attention with meta-learned in-context reasoning for few-shot tabular learning. Its encoder alternates standard, grouped, hierarchical, and relational attention, fusing their outputs through multi-CLS summarization to capture both local and global dependencies efficiently. A label-aware ICL head adapts on the fly and scales to large label spaces via hierarchical decision routing. Meta-trained on synthetically generated, structurally diverse tables with causal priors, Orion-Bix learns transferable inductive biases across heterogeneous data. Delivered as a scikit-learn compatible foundation model, it outperforms gradient-boosting baselines and remains competitive with state-of-the-art tabular foundation models on public benchmarks, showing that biaxial attention with episodic meta-training enables robust, few-shot-ready tabular learning. The model is publicly available at https://github.com/Lexsi-Labs/Orion-BiX .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。