让表格模型预测过程可解释，权重清晰可见。
Interpretable Tabular Foundation Models via In-Context Kernel Regression
- 用核函数显式替代预测层，使输出为训练标签的加权平均
- 在55个数据集上性能与现有模型持平，权重分布可量化分析
- 统一核方法、邻居法和注意力机制，适合需要可解释性的场景
像TabPFN和TabICL这样的表格基础模型通过上下文学习实现了顶尖性能，但其架构依然不透明。我们提出KernelICL框架，通过显式引入核函数（高斯、点积、kNN）替换最终预测层，使每个预测变为训练标签的透明加权平均，实现可量化的样本级可解释性。基于‘上下文学习等同于核回归’的洞察，我们构建了一个二维分类体系，形式化统一了传统核方法、现代邻近方法与注意力机制，并通过训练样本权重分布的困惑度来量化可检验性。在55个TALENT基准数据集上的实验表明，KernelICL性能与现有模型相当，证明在最终层施加显式核约束可在不牺牲性能的前提下实现可解释预测。
原文摘要 · Abstract (English)
Tabular foundation models like TabPFN and TabICL achieve state-of-the-art performance through in-context learning, yet their architectures remain fundamentally opaque. We introduce KernelICL, a framework to enhance tabular foundation models with quantifiable sample-based interpretability. Building on the insight that in-context learning is akin to kernel regression, we make this mechanism explicit by replacing the final prediction layer with kernel functions (Gaussian, dot-product, kNN) so that every prediction is a transparent weighted average of training labels. We introduce a two-dimensional taxonomy that formally unifies standard kernel methods, modern neighbor-based approaches, and attention mechanisms under a single framework, and quantify inspectability via the perplexity of the weight distribution over training samples. On 55 TALENT benchmark datasets, KernelICL achieves performance on par with existing tabular foundation models, demonstrating that explicit kernel constraints on the final layer enable inspectable predictions without sacrificing performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。