提出新方法让动作质量评估模型持续学习,避免遗忘旧知识。
Continual Action Quality Assessment via Adaptive Manifold-Aligned Graph Regularization
- 用自适应图正则化动态调整特征分布,稳定模型学习过程。
- 在线测试下相比最强基线提升12.2%相关性,效果显著。
- 适合需要长期更新的运动评分、康复训练等实际场景。
动作质量评估(AQA)用于量化视频中人类动作的质量,支撑体育评分、康复训练和技能评估等应用。真实场景中质量分布具有非平稳性,制约了传统方法的泛化能力。本文提出持续性AQA(CAQA),赋予AQA持续学习能力以应对分布演化并缓解灾难性遗忘。尽管参数高效微调在图像分类中表现良好,但对CAQA仍不足。实证与理论分析揭示:(i) 完全参数微调(FPFT)对有效表征学习必要;(ii) 无控制的FPFT导致过拟合与特征流形偏移,加剧遗忘。为此,提出自适应流形对齐图正则化(MAGR++),通过两步特征修复机制:流形投影器将历史特征映射至当前空间,图正则化器对齐局部与全局分布。基于三个数据集构建四个CAQA基准,设计定制评估协议与强基线,支持跨数据集系统比较。大量实验表明,MAGR++达到最优性能,离线平均相关性提升3.6%,在线提升12.2%,验证其鲁棒性与有效性。代码已开源。
原文摘要 · Abstract (English)
Action Quality Assessment (AQA) quantifies human actions in videos, supporting applications in sports scoring, rehabilitation, and skill evaluation. A major challenge lies in the non-stationary nature of quality distributions in real-world scenarios, which limits the generalization ability of conventional methods. We introduce Continual AQA (CAQA), which equips AQA with Continual Learning (CL) capabilities to handle evolving distributions while mitigating catastrophic forgetting. Although parameter-efficient fine-tuning of pretrained models has shown promise in CL for image classification, we find it insufficient for CAQA. Our empirical and theoretical analyses reveal two insights: (i) Full-Parameter Fine-Tuning (FPFT) is necessary for effective representation learning; yet (ii) uncontrolled FPFT induces overfitting and feature manifold shift, thereby aggravating forgetting. To address this, we propose Adaptive Manifold-Aligned Graph Regularization (MAGR++), which couples backbone fine-tuning that stabilizes shallow layers while adapting deeper ones with a two-step feature rectification pipeline: a manifold projector to translate deviated historical features into the current representation space, and a graph regularizer to align local and global distributions. We construct four CAQA benchmarks from three datasets with tailored evaluation protocols and strong baselines, enabling systematic cross-dataset comparison. Extensive experiments show that MAGR++ achieves state-of-the-art performance, with average correlation gains of 3.6% offline and 12.2% online over the strongest baseline, confirming its robustness and effectiveness. Our code is available at https://github.com/ZhouKanglei/MAGRPP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。