无需训练即可提升动作识别模型对新动作的泛化能力。
Boosting Skeleton-based Zero-Shot Action Recognition with Training-Free Test-Time Adaptation
- 用结构化骨架特征库实现推理时动态检索。
- 结合大模型语义推理，自动分配类别权重提升准确率。
- 适用于零样本和广义零样本场景，适合部署于新动作识别任务。
我们提出Skeleton-Cache，首个无需训练的测试时自适应框架，用于骨架基零样本动作识别（SZAR），旨在提升模型在推理阶段对未见动作的泛化能力。Skeleton-Cache将推理过程重构为轻量级检索任务，通过非参数化缓存存储结构化骨架表示，融合全局与细粒度局部描述子。为指导描述子预测的融合，利用大语言模型（LLMs）的语义推理能力，为不同类别分配特定重要性权重。结合结构化描述子与LLM引导的语义先验，Skeleton-Cache可在不进行额外训练或访问训练数据的情况下，动态适应未见动作。在NTU RGB+D 60/120与PKU-MMD II数据集上的大量实验表明，Skeleton-Cache在零样本与广义零样本设置下，持续提升了多种SZAR骨干模型的性能。代码已公开于https://github.com/Alchemist0754/Skeleton-Cache。
原文摘要 · Abstract (English)
We introduce Skeleton-Cache, the first training-free test-time adaptation framework for skeleton-based zero-shot action recognition (SZAR), aimed at improving model generalization to unseen actions during inference. Skeleton-Cache reformulates inference as a lightweight retrieval process over a non-parametric cache that stores structured skeleton representations, combining both global and fine-grained local descriptors. To guide the fusion of descriptor-wise predictions, we leverage the semantic reasoning capabilities of large language models (LLMs) to assign class-specific importance weights. By integrating these structured descriptors with LLM-guided semantic priors, Skeleton-Cache dynamically adapts to unseen actions without any additional training or access to training data. Extensive experiments on NTU RGB+D 60/120 and PKU-MMD II demonstrate that Skeleton-Cache consistently boosts the performance of various SZAR backbones under both zero-shot and generalized zero-shot settings. The code is publicly available at https://github.com/Alchemist0754/Skeleton-Cache.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。