arXiv:2505.11152cs.CV2025-05NeurIPS被引 11

解决手部接触估计中的数据不平衡问题,提升预测精度

Learning Dense Hand Contact Estimation from Imbalanced Data

  • 采用平衡采样策略,公平覆盖各类接触与非接触区域
  • 提出顶点级类别平衡损失,针对不同部位接触频率重加权
  • 适用于大规模手部交互数据,提升指尖外区域的泛化能力

手是人类互动的关键,研究手与外界的接触有助于全面理解其功能。近年来,手交互数据集不断增多,涵盖与物体、另一只手、场景和身体的交互。尽管任务重要性日益凸显且高质量数据增加,但如何有效学习密集手接触估计仍鲜有研究。主要挑战包括:数据集中多数区域无接触,存在类别不平衡;接触主要集中于手指尖,导致其他部位泛化困难。为此,本文提出从不平衡数据中学习密集手接触估计的框架(HACO)。为解决类别不平衡,引入平衡接触采样,构建多个采样组以均衡表示各类接触统计。为应对空间不平衡,提出顶点级类别平衡(VCB)损失,根据每个顶点在数据集中的接触频率,分别重加权其损失贡献。实验表明,该方法能有效利用大规模手接触数据,避免类别与空间不平衡问题。代码已开源。

原文摘要 · Abstract (English)

Hands are essential to human interaction, and exploring contact between hands and the world can promote comprehensive understanding of their function. Recently, there have been growing number of hand interaction datasets that cover interaction with object, other hand, scene, and body. Despite the significance of the task and increasing high-quality data, how to effectively learn dense hand contact estimation remains largely underexplored. There are two major challenges for learning dense hand contact estimation. First, there exists class imbalance issue from hand contact datasets where majority of regions are not in contact. Second, hand contact datasets contain spatial imbalance issue with most of hand contact exhibited in finger tips, resulting in challenges for generalization towards contacts in other hand regions. To tackle these issues, we present a framework that learns dense HAnd COntact estimation (HACO) from imbalanced data. To resolve the class imbalance issue, we introduce balanced contact sampling, which builds and samples from multiple sampling groups that fairly represent diverse contact statistics for both contact and non-contact vertices. Moreover, to address the spatial imbalance issue, we propose vertex-level class-balanced (VCB) loss, which incorporates spatially varying contact distribution by separately reweighting loss contribution of each vertex based on its contact frequency across dataset. As a result, we effectively learn to predict dense hand contact estimation with large-scale hand contact data without suffering from class and spatial imbalance issue. The codes are available at https://github.com/dqj5182/HACO_RELEASE.

手部接触数据不平衡深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。