通过自适应坐标调整,解决多人动作识别中骨架分布差异问题。
CHASE: Learning Convex Hull Adaptive Shift for Skeleton-based Multi-Entity Action Recognition
- 设计可学习的凸包约束位移网络,动态重定位骨架坐标系原点。
- 在6个数据集上提升单人模型在多人场景下的识别准确率。
- 适合需要提升多人交互动作识别性能的研究者使用。
基于骨架的多人动作识别任务旨在识别涉及多个不同实体的交互行为或群体活动。现有针对个体的模型在此任务中表现不佳,原因在于实体骨架间的固有分布差异导致主干网络优化不充分。为此,我们提出一种基于凸包自适应位移的多人动作识别方法(CHASE),以缓解实体间分布差距并消除后续主干网络的偏差。CHASE由可学习参数化网络和辅助目标构成:参数化网络通过两个关键组件实现样本自适应的骨架序列重定位;首先,隐式凸包约束自适应位移确保新坐标系原点位于骨架凸包内;其次,系数学习模块轻量化地参数化从骨架序列到凸组合系数的映射。此外,为引导该网络优化以最小化分布差异,我们提出小批量成对最大均值差异作为额外目标。CHASE作为一种样本自适应归一化方法,有效缓解了实体间分布差异,降低了数据偏差,提升了后续分类器在多人动作识别中的性能。在包括NTU Mutual 11/26、H2O、Assembly101、Collective Activity和Volleyball在内的六个数据集上,实验表明该方法能无缝适配单人骨干网络,并显著提升其在多人场景下的表现。代码已公开于 https://github.com/Necolizer/CHASE。
原文摘要 · Abstract (English)
Skeleton-based multi-entity action recognition is a challenging task aiming to identify interactive actions or group activities involving multiple diverse entities. Existing models for individuals often fall short in this task due to the inherent distribution discrepancies among entity skeletons, leading to suboptimal backbone optimization. To this end, we introduce a Convex Hull Adaptive Shift based multi-Entity action recognition method (CHASE), which mitigates inter-entity distribution gaps and unbiases subsequent backbones. Specifically, CHASE comprises a learnable parameterized network and an auxiliary objective. The parameterized network achieves plausible, sample-adaptive repositioning of skeleton sequences through two key components. First, the Implicit Convex Hull Constrained Adaptive Shift ensures that the new origin of the coordinate system is within the skeleton convex hull. Second, the Coefficient Learning Block provides a lightweight parameterization of the mapping from skeleton sequences to their specific coefficients in convex combinations. Moreover, to guide the optimization of this network for discrepancy minimization, we propose the Mini-batch Pair-wise Maximum Mean Discrepancy as the additional objective. CHASE operates as a sample-adaptive normalization method to mitigate inter-entity distribution discrepancies, thereby reducing data bias and improving the subsequent classifier's multi-entity action recognition performance. Extensive experiments on six datasets, including NTU Mutual 11/26, H2O, Assembly101, Collective Activity and Volleyball, consistently verify our approach by seamlessly adapting to single-entity backbones and boosting their performance in multi-entity scenarios. Our code is publicly available at https://github.com/Necolizer/CHASE .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。