用简单Transformer模型统一解决表格事件预测问题
A Simple Baseline for Predicting Events with Auto-Regressive Tabular Transformers
- 采用标准自回归Transformer与基础位置编码
- 在多个数据集上超越现有方法，支持多任务预测
- 适合需要标签预测、缺失值填补或序列建模的场景
许多现实世界中的表格数据应用需要利用历史事件来预测新事件的属性，例如判断信用卡交易是否欺诈，或预测客户对商品的评分。现有方法通常依赖昂贵、脆弱且特定于应用的技术，如时间感知的位置嵌入、学习得到的行和字段编码，以及处理类别不平衡的过采样方法。此外，这些方法常假设已知所有历史事件的标签，或仅预测预设标签而非数据特征本身。本文提出一种简单但灵活的基线方法：使用标准自回归大语言模型风格的Transformer，搭配基础位置编码和因果语言建模目标。该基线在多个流行数据集上表现优于现有方法，并可适用于多种使用场景。我们证明同一模型能同时完成标签预测、缺失值填补和事件序列建模。
原文摘要 · Abstract (English)
Many real-world applications of tabular data involve using historic events to predict properties of new ones, for example whether a credit card transaction is fraudulent or what rating a customer will assign a product on a retail platform. Existing approaches to event prediction include costly, brittle, and application-dependent techniques such as time-aware positional embeddings, learned row and field encodings, and oversampling methods for addressing class imbalance. Moreover, these approaches often assume specific use-cases, for example that we know the labels of all historic events or that we only predict a pre-specified label and not the data's features themselves. In this work, we propose a simple but flexible baseline using standard autoregressive LLM-style transformers with elementary positional embeddings and a causal language modeling objective. Our baseline outperforms existing approaches across popular datasets and can be employed for various use-cases. We demonstrate that the same model can predict labels, impute missing values, or model event sequences.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。