用少量样本高效适配大模型做动作检测，提升精度与泛化能力。
Group Relative Augmentation for Data Efficient Action Detection
- 通过可学习的内部特征增强生成多样特征，适配小样本任务。
- 在AVA和MOMA数据集上实现高mAP，仅用少量样本即达良好效果。
- 适合需要小样本高效微调的动作识别研究者使用。
将大型视频-语言模型（VLM）用于动作检测时，仅用少量示例面临过拟合及场景级预训练与人物中心理解之间的粒度不匹配问题。本文提出一种高效适配策略，结合参数高效微调（LoRA）与一种新颖的可学习内部特征增强方法。该增强在冻结的VLM主干中通过FiLM实现，生成直接相关的多样化特征。此外，引入分组加权损失函数，根据每个增强样本预测偏离组平均值的程度动态调整其训练贡献，优先选择信息量大且合理的增强。在复杂多标签、多人动作检测数据集（AVA、MOMA）上验证了该方法的有效性，实现了优异的mAP表现，并展现出显著的数据效率，仅需少量示例即可有效适配VLM。
原文摘要 · Abstract (English)
Adapting large Video-Language Models (VLMs) for action detection using only a few examples poses challenges like overfitting and the granularity mismatch between scene-level pre-training and required person-centric understanding. We propose an efficient adaptation strategy combining parameter-efficient tuning (LoRA) with a novel learnable internal feature augmentation. Applied within the frozen VLM backbone using FiLM, these augmentations generate diverse feature variations directly relevant to the task. Additionally, we introduce a group-weighted loss function that dynamically modulates the training contribution of each augmented sample based on its prediction divergence relative to the group average. This promotes robust learning by prioritizing informative yet reasonable augmentations. We demonstrate our method's effectiveness on complex multi-label, multi-person action detection datasets (AVA, MOMA), achieving strong mAP performance and showcasing significant data efficiency for adapting VLMs from limited examples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。