CRUMB通过匹配分布的分批策略，让大表数据推理更快更准。
CRUMB: Efficient Prior Fitted Network Inference via Distributionally Matched Context Batching
- 按测试样本聚类后，用MMD最小化选出匹配的训练子集
- 在TabArena上3种模型均优于现有方法，精度不降且提速显著
- 无需重训练，适合应对数据分布变化的实时推理场景
先验拟合网络（PFNs）是一类能够进行上下文学习的表格基础模型，将整个带标签训练集作为上下文，在单次前向传播中生成测试查询的预测。然而，许多PFN架构中的自注意力机制呈二次增长，导致在大规模训练数据下推理变得不可行。我们提出CRUMB（基于最小化最大均值差异的聚类检索分批），一种三阶段推理封装：(i) 对测试查询进行聚类；(ii) 通过贪心最小化最大均值差异（MMD）为每个簇选择一个小而分布匹配的训练子集；(iii) 在每个缩减后的上下文批次上运行精确的PFN推理。CRUMB与架构无关，无需重新训练。在包含51个数据集的TabArena基准上，对三种PFN架构（TabPFNv2、TabICLv1、TabICLv2）评估显示，CRUMB优于现有的先进上下文选择策略。同时，我们证明了CRUMB对协变量漂移具有鲁棒性，因为MMD最小化步骤能自然地使训练上下文分布与当前测试批次分布对齐。
原文摘要 · Abstract (English)
Prior-fitted networks (PFNs) are a promising class of tabular foundation models that perform in-context learning, whereby the entire labelled training set is supplied as context, and predictions for test queries are produced in a single forward pass. However, the quadratically scaling self-attention mechanism in many PFN architectures makes inference prohibitive for very large training datasets. We propose CRUMB (Clustered Retrieval Using Minimised-MMD Batching), a three-stage inference wrapper that (i) clusters the test queries, (ii) selects a small, distributionally matched training subset for each cluster by greedily minimising the maximum mean discrepancy (MMD), and (iii) runs exact PFN inference on each reduced-context batch. CRUMB is architecture-agnostic and requires no retraining. On the 51-dataset TabArena benchmark, evaluated across three PFN architectures (TabPFNv2, TabICLv1, TabICLv2), we show that CRUMB outperforms similar state-of-the-art context selection strategies. We also show that CRUMB is resilient to covariate drift, as the MMD-minimisation step naturally helps align the training context distribution to match the current test batch distributions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。