arXiv:2602.22586cs.LGcs.AI2026-02

用统一模型同时生成表格中的数字、类别和自由文本,解决跨模态生成难题。

TabDLM: Free-Form Tabular Data Generation via Joint Numerical-Language Diffusion

  • 基于掩码扩散语言模型,联合建模文本、类别与数值特征。
  • 在多个基准上生成的表格数据在真实性和一致性上优于现有方法。
  • 适合需要高质量合成表格数据的研究者,尤其关注医疗或评论类数据。

由于对数据增强、基础模型和隐私保护的重要性,合成表格数据生成受到越来越多关注。然而,现实世界中的表格数据越来越多地包含自由文本字段(如评论或临床笔记)以及结构化的数值和类别属性。联合建模不同模态的异构表格生成仍具挑战性。现有方法主要分为两类:基于扩散的方法和基于大语言模型(LLM)的方法。扩散模型能在连续或离散空间中捕捉数值和类别特征的复杂依赖关系,但将其扩展到开放式文本时面临困难,常导致文本质量下降。相比之下,基于LLM的生成器能自然产出流畅文本,但其离散分词会扭曲精确或范围广泛的数值,影响数字与语言的准确建模。本文提出TabDLM,一种通过掩码扩散语言模型(MDLM)构建的统一框架,实现自由文本表格数据的联合数值-语言扩散生成。TabDLM通过掩码扩散建模文本与类别特征,利用学习的专用数值令牌嵌入对数值特征进行连续扩散过程;双向注意力机制在单一模型中捕获跨模态交互。在多种基准上的实验表明,相比强大的扩散和基于LLM的基线方法,TabDLM表现出显著优势。

原文摘要 · Abstract (English)

Synthetic tabular data generation has attracted growing attention due to its importance for data augmentation, foundation models, and privacy. However, real-world tabular datasets increasingly contain free-form text fields (e.g., reviews or clinical notes) alongside structured numerical and categorical attributes. Generating such heterogeneous tables with joint modeling of different modalities remains challenging. Existing approaches broadly fall into two categories: diffusion-based methods and LLM-based methods. Diffusion models can capture complex dependencies over numerical and categorical features in continuous or discrete spaces, but extending them to open-ended text is nontrivial and often leads to degraded text quality. In contrast, LLM-based generators naturally produce fluent text, yet their discrete tokenization can distort precise or wide-range numerical values, hindering accurate modeling of both numbers and language. In this work, we propose TabDLM, a unified framework for free-form tabular data generation via a joint numerical-language diffusion model built on masked diffusion language models (MDLMs). TabDLM models textual and categorical features through masked diffusion, while modeling numerical features with a continuous diffusion process through learned specialized numeric tokens embedding; bidirectional attention then captures cross-modality interactions within a single model. Extensive experiments on diverse benchmarks demonstrate the effectiveness of TabDLM compared to strong diffusion- and LLM-based baselines.

表格生成扩散模型多模态合成数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。