用大模型直接做表格数据联合概率预测，无需预处理
JoLT: Joint Probabilistic Predictions on Tabular Data Using LLMs
- 基于大模型上下文学习能力，直接建模多变量联合分布
- 低样本下分类回归任务表现优于现有方法，支持异构数据类型
- 自动处理缺失值并实现文本引导的插补，适合实际应用
我们提出一种基于大语言模型（LLM）的表格数据联合概率预测方法——JoLT（Joint LLM Process for Tabular data）。JoLT利用LLM的上下文学习能力，在用户指定的问题侧信息条件下，定义表格数据的联合分布，从而利用模型中编码的海量潜在相关知识。该方法无需数据转换、预处理、特殊缺失值处理或模型训练，即可对具有异构数据类型的多个目标变量建立联合分布。实验表明，JoLT在低样本场景下的单目标与多目标表格分类和回归任务中均优于对比方法。此外，我们证明其可通过文本侧信息自动处理缺失数据并实现数据插补。由于方法简单且通用，我们认为JoLT适用于多种真实预测问题。
原文摘要 · Abstract (English)
We introduce a simple method for probabilistic predictions on tabular data based on Large Language Models (LLMs) called JoLT (Joint LLM Process for Tabular data). JoLT uses the in-context learning capabilities of LLMs to define joint distributions over tabular data conditioned on user-specified side information about the problem, exploiting the vast repository of latent problem-relevant knowledge encoded in LLMs. JoLT defines joint distributions for multiple target variables with potentially heterogeneous data types without any data conversion, data preprocessing, special handling of missing data, or model training, making it accessible and efficient for practitioners. Our experiments show that JoLT outperforms competitive methods on low-shot single-target and multi-target tabular classification and regression tasks. Furthermore, we show that JoLT can automatically handle missing data and perform data imputation by leveraging textual side information. We argue that due to its simplicity and generality, JoLT is an effective approach for a wide variety of real prediction problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。