用局部邻近分析选最优样本，让表格预测更省资源
ARASH: Adaptive Retrieval And Shot Selection for Tabular Prediction
- 基于训练集局部邻近性动态选最相关样本作为提示
- 使TabPFN的提示长度减少1261.5倍，内存降低2.56倍
- 适合需要低资源部署的表格预测场景
表格预测在众多应用中至关重要。大语言模型的成功催生了多种将其适配到表格领域的方案。主流方法是训练或微调专用的表格基础模型（TFM），如TabPFN，但这类模型需大量计算资源，频繁重训练不切实际。上下文学习（ICL），特别是少样本提示，提供了一种资源高效的替代方案。然而，如何为表格数据识别最相关的行作为提示样本仍具挑战。本文提出ARASH（自适应、查询特定的检索与样本选择），通过分析训练集中局部邻近性来选取最优提示样本，提升TFM效率。实验表明，ARASH使TabPFN的提示长度减少1261.5倍，内存使用降低2.56倍，同时保持相当的预测精度。
原文摘要 · Abstract (English)
Tabular prediction is a critical task across numerous applications. The recent success of large language models has sparked various approaches for adapting them to the tabular domain. A prevalent strategy involves training or fine-tuning specialized Tabular Foundation Models (TFMs) such as TabPFN. However, TFMs require substantial computational resources, and frequent model retraining is often impractical. In-context learning (ICL), specifically, few-shot prompting, offers a resource-efficient alternative to enhance performance. Yet, identifying the most relevant rows to serve as shots remains a challenge for tabular data. This paper introduces ARASH (Adaptive, query-specific Retrieval And Shot selection), a method that improves TFM efficiency by selecting optimal shots based on local neighborhood analysis within the training set. Our results demonstrate that ARASH reduces the prompt length and memory usage of TabPFN by 1261.5$\times$ and 2.56$\times$, respectively, while providing comparable accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。