arXiv:2507.17008cs.CVcs.AI2025-07

用生成模型缓解手形识别数据不平衡问题,提升分类准确率5%。

Bringing Balance to Hand Shape Classification: Mitigating Data Imbalance Through Generative Models

  • 采用条件GAN生成真实感手形图像,结合真实数据训练分类器。
  • 在RWTH数据集上实现5%的准确率提升,优于当前最优方法。
  • 可跨数据集泛化,无需重新训练生成器,适合多语言手语研究。

大多数手语手形数据集规模小且严重失衡,给模型训练带来挑战。本文探索通过生成合成数据来增强手形分类器的训练数据。以小而失衡的RWTH德语手语手形数据集为基础,训练EfficientNet分类器,并采用不同策略融合生成与真实图像。对比两种生成对抗网络(GAN)架构:ReACGAN利用辅助分类器将标签信息引入生成过程,生成与特定手形标签一致的图像;SPADE则通过空间自适应归一化,基于姿态信息生成空间配置准确的图像。实验表明,所提方法在RWTH数据集上使当前最先进准确率提升5%,有效缓解小样本与数据不平衡问题。此外,该方法具备跨数据集泛化能力,仅需在大型HaGRID数据集上训练姿态生成模型,即可在不重训练生成器的情况下达到与单源训练分类器相当的性能。

原文摘要 · Abstract (English)

Most sign language handshape datasets are severely limited and unbalanced, posing significant challenges to effective model training. In this paper, we explore the effectiveness of augmenting the training data of a handshape classifier by generating synthetic data. We use an EfficientNet classifier trained on the RWTH German sign language handshape dataset, which is small and heavily unbalanced, applying different strategies to combine generated and real images. We compare two Generative Adversarial Networks (GAN) architectures for data generation: ReACGAN, which uses label information to condition the data generation process through an auxiliary classifier, and SPADE, which utilizes spatially-adaptive normalization to condition the generation on pose information. ReACGAN allows for the generation of realistic images that align with specific handshape labels, while SPADE focuses on generating images with accurate spatial handshape configurations. Our proposed techniques improve the current state-of-the-art accuracy on the RWTH dataset by 5%, addressing the limitations of small and unbalanced datasets. Additionally, our method demonstrates the capability to generalize across different sign language datasets by leveraging pose-based generation trained on the extensive HaGRID dataset. We achieve comparable performance to single-source trained classifiers without the need for retraining the generator.

手语识别数据生成不平衡数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。