通过融合姿态估计中间特征，提升零样本骨骼动作识别性能。
PoseBridge: Bridging the Skeletonization Gap for Zero-Shot Skeleton-Based Action Recognition
- 在姿态估计阶段提取语义线索，桥接视觉与骨骼表征
- 在Kinetics-200/400上相比最强基线提升13.3-17.4分
- 适合关注跨模态对齐与零样本动作识别的研究者
零样本骨骼动作识别（ZSSAR）通常被视为骨骼与文本对齐问题：编码关节坐标序列，与语言对齐并分类未见动作。我们指出，这种对齐往往过晚。骨骼是人体姿态估计（HPE）的压缩输出，对齐开始时，人-物交互和姿态相关视觉线索可能已丢失。我们称此为上游语义损失。为此，提出PoseBridge框架，利用HPE过程中的中间表示，将语义线索从原始姿态估计传递至骨骼-文本对齐。不添加RGB分支或物体检测器，而是从同一HPE流程中提取姿态锚定语义线索，经骨骼条件桥接与语义原型适应进行传递。在NTU-RGB+D 60/120、PKU-MMD及Kinetics-200/400上均提升性能。在包含复杂场景与动作上下文的Kinetics-200/400 PURLS基准上，提升最显著，优于最强基线13.3–17.4点，覆盖全部八组划分。代码将公开发布。
原文摘要 · Abstract (English)
Zero-shot skeleton-based action recognition (ZSSAR) is typically treated as a skeleton-text alignment problem: encode joint-coordinate sequences, align them with language, and classify unseen actions. We argue that this alignment is often too late. Skeletons are not complete action observations, but compressed outputs of human pose estimation (HPE); by the time alignment begins, human-object interactions and pose-relative visual cues may no longer be explicit. We call this upstream semantic loss. To address it, we propose PoseBridge, an HPE-aware ZSSAR framework that bridges intermediate HPE representations to skeleton-text alignment. Rather than adding an RGB action branch or object detector, PoseBridge extracts pose-anchored semantic cues from the same HPE process that produces skeletons, then transfers them through skeleton-conditioned bridging and semantic prototype adaptation. Across NTU-RGB+D 60/120, PKU-MMD, and Kinetics-200/400, PoseBridge improves ZSSAR performance under the evaluated protocols. On the Kinetics-200/400 PURLS benchmark, which contains in-the-wild videos with diverse scenes and action contexts, PoseBridge shows the clearest separation, improving the strongest compared baseline by 13.3-17.4 points across all eight splits. Our code will be publicly released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。