通过融合视觉、姿态与骨骼图结构，提升细粒度动作识别准确率。
Fine-Grained Action Recognition with Cross-Attentive Latent Sparse Experts
- 分三路提取视觉、姿态热图和骨骼拓扑特征，用交叉注意力对齐信息。
- 在Gym288上将平均准确率从68.6%提升至76.2%，增益7.6点。
- 无需文本监督或大规模预训练，适合动作识别与人体运动分析场景。
细粒度人体动作识别（FHAR）需区分外观相似但身体姿态、时间或局部细节不同的动作。RGB表示保留视觉上下文但常忽略关节几何，骨架表示编码运动学却丢失密集空间信息。本文提出FineX，将细粒度线索分解为RGB外观、姿态热图几何与骨骼图拓扑。通过成对交叉注意力实现对称且流保持的信息交换，随后采用流内潜在稀疏专家混合模型，使每路表示路由至内容相关的共享专家子集，并以负载均衡目标进行正则化。FineX在Gym99、Gym288和Diving48上取得当前最优性能。在长尾分布的Gym288上，平均类别准确率从68.6%提升至76.2%（+7.6点），无需文本监督或大规模视觉-语言预训练，验证了结构化视觉-姿态-图融合与条件专家优化在FHAR中的有效性。
原文摘要 · Abstract (English)
Fine-grained human action recognition (FHAR) must distinguish visually similar actions that differ mainly in body configuration, timing, or local appearance. RGB representations retain visual context but often suppress joint-level geometry, whereas skeleton representations encode kinematics but discard dense spatial detail. We introduce FineX, which factorizes fine-grained cues into RGB appearance, pose heatmap geometry, and skeletal-graph topology. Pairwise cross-attention enables symmetric, stream-preserving information exchange, followed by a streamwise latent sparse Mixture-of-Experts that routes each representation to a content-dependent subset of shared experts, regularized by a load-balancing objective. FineX achieves state-of-the-art results on Gym99, Gym288, and Diving48. On the long-tailed Gym288, it raises mean class accuracy from 68.6% to 76.2% (+7.6 points) without textual supervision or large-scale vision-language pre-training, demonstrating the benefit of structured visual-pose-graph fusion and conditional expert refinement for FHAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。