arXiv:2501.06863cs.LGcs.AI2025-01被引 7

用大模型微调实现表格数据跨任务迁移,效果超越传统方法。

Transfer Learning of Tabular Data by Finetuning Large Language Models

  • 直接微调大语言模型处理表格分类任务
  • 在特征少于10个的数据集上表现优于现有方法
  • 计算成本低,适合资源有限场景

尽管人工智能取得突破,深度学习在表格数据上的表现仍受限于异构特征空间和样本量不足,缺乏有效的迁移学习方案。生成式AI时代的大语言模型(LLM)为多样数据与领域带来前所未有的学习机会。本文研究了LLM API与基于微调的迁移学习在表格数据分类中的应用。其中,LLM API通过文本提示生成标记化数据与指令,而迁移学习则对预训练模型进行微调以适应目标分类任务。本文提出一种端到端的LLM微调方法,在无预训练表格数据模型的情况下,实现了对十个基准数据集的跨数据迁移学习。该方法在特征少于10个的表格数据集上,性能超越当前最优机器学习与深度学习方法。其计算开销仅为其他深度学习或API方案的一小部分,同时保持了竞争力甚至更优的分类表现。

原文摘要 · Abstract (English)

Despite the artificial intelligence (AI) revolution, deep learning has yet to achieve much success with tabular data due to heterogeneous feature space and limited sample sizes without viable transfer learning. The new era of generative AI, powered by large language models (LLM), brings unprecedented learning opportunities to diverse data and domains. This paper investigates the effectiveness of an LLM application programming interface (API) and transfer learning of LLM in tabular data classification. LLM APIs respond to input text prompts with tokenized data and instructions, whereas transfer learning finetunes an LLM for a target classification task. This paper proposes an end-to-end finetuning of LLM to demonstrate cross-data transfer learning on ten benchmark data sets when large pre-trained tabular data models do not exist to facilitate transfer learning. The proposed LLM finetuning method outperforms state-of-the-art machine and deep learning methods on tabular data with less than ten features - a standard feature size for tabular data sets. The transfer learning approach uses a fraction of the computational cost of other deep learning or API-based solutions while ensuring competitive or superior classification performance.

表格数据大模型迁移学习微调

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。