arXiv:2510.09783cs.LGcs.AI2025-10被引 4

用大模型生成更多样、更真实的少数类样本,提升不平衡分类效果

Large Language Models for Imbalanced Classification: Diversity makes the difference

  • 结合标签与特征条件生成样本,增强多样性
  • 通过插值样本微调模型,显著提升生成样本变异性
  • 在10个数据集上超越8种顶尖方法,适合数据不平衡场景

过采样是解决不平衡分类问题的常用方法,核心思想是生成额外的少数类样本以平衡数据集。现有方法如SMOTE需将类别变量转为数值向量,常导致信息丢失。近期基于大语言模型(LLM)的方法虽能克服此限制,但生成的少数类样本多样性不足,影响下游分类任务的鲁棒性与泛化能力。为此,本文提出一种新型LLM-based过采样方法,以提升生成样本的多样性:首先,设计一种同时依赖少数类标签和特征的采样策略;其次,开发新的微调预训练LLM的排列策略;第三,不仅在少数类样本上微调,还在插值样本上进行微调,进一步丰富变异性。在10个表格数据集上的大量实验表明,该方法显著优于8种SOTA基线。生成的合成样本既真实又多样。此外,我们从熵的角度提供理论分析,证明该方法能有效促进生成样本的多样性。

原文摘要 · Abstract (English)

Oversampling is one of the most widely used approaches for addressing imbalanced classification. The core idea is to generate additional minority samples to rebalance the dataset. Most existing methods, such as SMOTE, require converting categorical variables into numerical vectors, which often leads to information loss. Recently, large language model (LLM)-based methods have been introduced to overcome this limitation. However, current LLM-based approaches typically generate minority samples with limited diversity, reducing robustness and generalizability in downstream classification tasks. To address this gap, we propose a novel LLM-based oversampling method designed to enhance diversity. First, we introduce a sampling strategy that conditions synthetic sample generation on both minority labels and features. Second, we develop a new permutation strategy for fine-tuning pre-trained LLMs. Third, we fine-tune the LLM not only on minority samples but also on interpolated samples to further enrich variability. Extensive experiments on 10 tabular datasets demonstrate that our method significantly outperforms eight SOTA baselines. The generated synthetic samples are both realistic and diverse. Moreover, we provide theoretical analysis through an entropy-based perspective, proving that our method encourages diversity in the generated samples.

大模型不平衡分类过采样多样性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。