让表格模型处理超长数据，无需压缩预处理。
Chunked TabPFN: Exact Training-Free In-Context Learning for Long-Context Tabular Data
- 分块计算注意力，突破1万条上下文限制
- 在TabArena上性能超越树模型，保持高精度
- 兼容普通显卡，适合真实长表场景
TabPFN v2在多个表格基准测试中表现优于传统树模型，这很不寻常，因为树模型通常在表格数据上表现最强。然而，其无法处理超过1万条上下文标记，因Transformer具有二次计算和内存开销。不同于依赖上下文压缩的现有方法（如通过K近邻选择代表性样本），我们引入一种分块块策略，在TabPFN框架内计算注意力。该设计兼容标准GPU设置，据我们所知，是首个使TabPFN在无任何预处理的情况下处理长上下文的方法。我们在标准的TabArena基准上验证了该方法的有效性，代码已开源：https://github.com/mrsergazinov/chunk_tabpfn。
原文摘要 · Abstract (English)
TabPFN v2 achieves better results than tree-based models on several tabular benchmarks, which is notable since tree-based models are usually the strongest choice for tabular data. However, it cannot handle more than 10K context tokens because transformers have quadratic computation and memory costs. Unlike existing approaches that rely on context compression, such as selecting representative samples via K-nearest neighbors (KNN), we introduce a tiled-block strategy to compute attention within the TabPFN framework. This design is compatible with standard GPU setups and, to the best of our knowledge, is the first to enable TabPFN to process long contexts without any pre-processing. We demonstrate the effectiveness of our approach on the standard TabArena benchmark, with code available at https://github.com/mrsergazinov/chunk_tabpfn.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。