解决长视频动作质量评估中的领域偏移问题,提升模型泛化能力。
PHI: Bridging Domain Shift in Long-Term Action Quality Assessment via Progressive Hierarchical Instruction
- 分层次设计指令机制,逐步缩小特征与任务目标间的域差距。
- 在三个数据集上达到最优性能,显著优于现有方法。
- 适合需要高精度长视频动作评估的场景,如体育训练分析。
长期动作质量评估(AQA)旨在量化评估长视频中动作的表现。然而,现有方法受限于预训练大规模动作识别主干网络与特定AQA任务之间的领域偏移,导致性能受限。由于在小规模AQA数据集上微调资源密集型主干网络不切实际,本文识别出两个层面的领域偏移:任务级(任务目标差异)和特征级(关键特征差异)。针对更严重的特征级偏移,提出渐进式分层指令(PHI)框架,包含两种策略:一是间隙最小化流(GMF),利用光流匹配逐步学习从浅层到深层的快速特征映射路径,以缩小初始特征与目标特征间的域差距;二是时间增强注意力模块,捕捉长时依赖关系,对AQA至关重要。二是列表级对比正则化(LCR),通过成对比较实现粗粒度到细粒度的对齐,学习精细线索并缓解域偏移。整合后,PHI在三个代表性长期AQA数据集上取得当前最优表现,验证了其在应对长期AQA中领域偏移问题上的有效性。
原文摘要 · Abstract (English)
Long-term Action Quality Assessment (AQA) aims to evaluate the quantitative performance of actions in long videos. However, existing methods face challenges due to domain shifts between the pre-trained large-scale action recognition backbones and the specific AQA task, thereby hindering their performance. This arises since fine-tuning resource-intensive backbones on small AQA datasets is impractical. We address this by identifying two levels of domain shift: task-level, regarding differences in task objectives, and feature-level, regarding differences in important features. For feature-level shifts, which are more detrimental, we propose Progressive Hierarchical Instruction (PHI) with two strategies. First, Gap Minimization Flow (GMF) leverages flow matching to progressively learn a fast flow path that reduces the domain gap between initial and desired features across shallow to deep layers. Additionally, a temporally-enhanced attention module captures long-range dependencies essential for AQA. Second, List-wise Contrastive Regularization (LCR) facilitates coarse-to-fine alignment by comprehensively comparing batch pairs to learn fine-grained cues while mitigating domain shift. Integrating these modules, PHI offers an effective solution. Experiments demonstrate that PHI achieves state-of-the-art performance on three representative long-term AQA datasets, proving its superiority in addressing the domain shift for long-term AQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。