arXiv:2409.16118cs.LG2024-09NeurIPS被引 18

为小样本表格数据设计了分类型能量模型,生成更高质量的合成数据。

TabEBM: A Tabular Data Augmentation Method with Distinct Class-Specific Energy-Based Models

  • 为每类数据单独构建能量模型,避免共享模型过拟合。
  • 在多种小规模数据集上,合成数据提升分类准确率最高达12.3%。
  • 适合医疗、化学等小样本高价值领域使用。

医学、物理和化学等关键领域常面临数据收集困难,导致分类模型在小样本数据上表现不佳。为提升性能,通常通过生成合成数据进行数据增强,但现有表格生成方法多学习联合分布或类条件分布,易在小数据上过拟合,生成数据质量差,反而降低分类效果。为此,我们提出TabEBM,一种基于能量模型(EBM)的新型类条件生成方法。与以往使用单一模型建模所有类条件分布不同,其核心创新是为每个类别分别构建独立的能量模型,分别捕捉各类别特有的数据分布。该方法在模糊类别分布下仍能建立稳健的能量势场。实验表明,TabEBM生成的合成数据在质量和统计保真度上均优于现有方法。用于数据增强时,其合成数据在多种不同规模的数据集上持续提升分类性能,尤其在小样本情况下表现显著,最高提升达12.3%。代码已开源:https://github.com/andreimargeloiu/TabEBM。

原文摘要 · Abstract (English)

Data collection is often difficult in critical fields such as medicine, physics, and chemistry. As a result, classification methods usually perform poorly with these small datasets, leading to weak predictive performance. Increasing the training set with additional synthetic data, similar to data augmentation in images, is commonly believed to improve downstream classification performance. However, current tabular generative methods that learn either the joint distribution $ p(\mathbf{x}, y) $ or the class-conditional distribution $ p(\mathbf{x} \mid y) $ often overfit on small datasets, resulting in poor-quality synthetic data, usually worsening classification performance compared to using real data alone. To solve these challenges, we introduce TabEBM, a novel class-conditional generative method using Energy-Based Models (EBMs). Unlike existing methods that use a shared model to approximate all class-conditional densities, our key innovation is to create distinct EBM generative models for each class, each modelling its class-specific data distribution individually. This approach creates robust energy landscapes, even in ambiguous class distributions. Our experiments show that TabEBM generates synthetic data with higher quality and better statistical fidelity than existing methods. When used for data augmentation, our synthetic data consistently improves the classification performance across diverse datasets of various sizes, especially small ones. Code is available at https://github.com/andreimargeloiu/TabEBM.

表格生成能量模型数据增强小样本学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。