通过动态调整视觉语义对齐,提升骨骼动作零样本识别准确率
DynaPURLS: Dynamic Refinement of Part-Aware Representations for Skeleton-Based Zero-Shot Action Recognition

- 用大语言模型生成分层文本描述,捕捉全局动作与局部肢体动态
- 引入自适应分区模块,按语义分组关节以生成细粒度视觉表征
- 推理时动态优化文本特征,适合跨域迁移的零样本动作识别任务
零样本骨骼动作识别(ZS-SAR)受限于现有方法依赖静态类别语义对齐骨架特征。这种粗粒度对齐难以弥合已见与未见类之间的领域差异,阻碍细粒度视觉知识的有效迁移。为此,本文提出统一框架DynaPURLS,建立鲁棒的多尺度视觉-语义对应关系,并在推理时动态精炼以增强泛化能力。该框架利用大语言模型生成包含全局运动与局部肢体动态的分层文本描述;同时,自适应分区模块通过语义分组生成细粒度骨架表征。为应对训练-测试阶段的领域偏移,DynaPURLS引入动态精炼模块:推理时通过轻量级可学习投影将文本特征适配至输入视觉流,其稳定性由置信度感知、类别平衡的记忆库保障,缓解噪声伪标签引发的误差传播。在三个大规模基准数据集NTU RGB+D 60/120和PKU-MMD上的大量实验表明,DynaPURLS显著优于现有方法,创下新最佳性能记录。代码已开源于https://github.com/Alchemist0754/DynaPURLS。
原文摘要 · Abstract (English)
Zero-shot skeleton-based action recognition (ZS-SAR) is fundamentally constrained by prevailing approaches that rely on aligning skeleton features with static, class-level semantics. This coarse-grained alignment fails to bridge the domain shift between seen and unseen classes, thereby impeding the effective transfer of fine-grained visual knowledge. To address these limitations, we introduce \textbf{DynaPURLS}, a unified framework that establishes robust, multi-scale visual-semantic correspondences and dynamically refines them at inference time to enhance generalization. Our framework leverages a large language model to generate hierarchical textual descriptions that encompass both global movements and local body-part dynamics. Concurrently, an adaptive partitioning module produces fine-grained visual representations by semantically grouping skeleton joints. To fortify this fine-grained alignment against the train-test domain shift, DynaPURLS incorporates a dynamic refinement module. During inference, this module adapts textual features to the incoming visual stream via a lightweight learnable projection. This refinement process is stabilized by a confidence-aware, class-balanced memory bank, which mitigates error propagation from noisy pseudo-labels. Extensive experiments on three large-scale benchmark datasets, including NTU RGB+D 60/120 and PKU-MMD, demonstrate that DynaPURLS significantly outperforms prior art, setting new state-of-the-art records. The source code is made publicly available at https://github.com/Alchemist0754/DynaPURLS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。