用树结构增强Transformer，更高效生成保持真实特性的表格数据。
TabTreeFormer: Tabular Data Generation Using Hybrid Tree-Transformer
- 融合树模型的先验知识，更好处理离散弱相关特征。
- 新分词器降低序列长度与词汇量，提升效率且不损性能。
- 在9个数据集上优于8个基线模型，实用性和隐私性俱佳。
Transformer在表格数据生成中表现优异，但缺乏对表格数据内在特性的领域特定归纳偏置，且因二次计算复杂度导致扩展性与效率差。本文提出TabTreeFormer，一种混合树-Transformer架构，引入树模型的先验特性（如非光滑性、非旋转不变性），以有效处理表格数据中的离散与弱相关特征。为提升数值保真度并捕捉多模态分布，我们设计了一种新型分词器，根据表格值的复杂度学习词元序列，减少词汇量与序列长度，实现更紧凑高效的表示，同时保持性能。我们在9个多样化数据集上评估了TabTreeFormer，对比8种生成模型。结果表明，其在实用性、保真度和隐私性指标上持续优于基线，且效率具竞争力。尤其在注重数据实用性的场景下，最优变体相比基线提升44%性能。代码已开源。
原文摘要 · Abstract (English)
Transformers have shown impressive results in tabular data generation. However, they lack domain-specific inductive biases which are critical for preserving the intrinsic characteristics of tabular data. They also suffer from poor scalability and efficiency due to quadratic computational complexity. In this paper, we propose TabTreeFormer, a hybrid transformer architecture that integrates inductive biases of tree-based models (e.g., non-smoothness and non-rotational invariance) to effectively handle the discrete and weakly correlated features in tabular datasets. To improve numerical fidelity and capture multimodal distributions, we introduce a novel tokenizer that learns token sequences based on the complexity of tabular values. This reduces vocabulary size and sequence length, yielding more compact and efficient representations without sacrificing performance. We evaluate TabTreeFormer on nine diverse datasets, benchmarking against eight generative models. We show that TabTreeFormer consistently outperforms baselines in utility, fidelity, and privacy metrics with competitive efficiency. Notably, in scenarios prioritizing data utility over privacy and efficiency, the best variant of TabTreeFormer delivers a 44% performance gain relative to its baseline variant. Our code is available at: https://github.com/li-jiayu-ljy/tabtreeformer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。