对比表格数据的通用嵌入模型，发现传统特征工程更高效
Comparing Task-Agnostic Embedding Models for Tabular Data
- 用经典特征工程提取通用嵌入，替代复杂基础模型
- 在异常检测和监督学习任务中表现相当或更好
- 计算开销远低于大模型，适合资源受限场景
近期针对表格数据的基础模型通过上下文学习实现了强大的特定任务性能。然而，它们将表征学习与任务特异性推理封装在单一、高资源消耗的网络中。本文聚焦于表征学习，即可迁移的、任务无关的嵌入。我们系统评估了从表格基础模型（TabPFN、TabICL 和 TabSTAR）中提取的任务无关表示，以及经典特征工程方法（TableVectorizer 和球形模型）在多种应用任务中的表现，包括异常检测（ADBench）和监督学习（TabArena Lite）。结果表明，简单特征工程方法在性能上达到或超过表格基础模型，同时显著降低计算资源需求。
原文摘要 · Abstract (English)
Recent foundation models for tabular data achieve strong task-specific performance via in-context learning. Nevertheless, they focus on direct prediction by encapsulating both representation learning and task-specific inference inside a single, resource-intensive network. This work specifically focuses on representation learning, i.e., on transferable, task-agnostic embeddings. We systematically evaluate task-agnostic representations extracted from tabular foundation models (TabPFN, TabICL and TabSTAR) alongside classical feature engineering (TableVectorizer and a sphere model) across a variety of application tasks as outlier detection (ADBench) and supervised learning (TabArena Lite). We find that simple feature engineering methods achieve comparable or superior performance while requiring significantly less computational resources than tabular foundation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。