针对第一人称动作检测,提出可靠性感知的动态融合方法。
EgoAction: Egocentric Action Composition with Reliability-Aware Temporal Fusion for the EPIC-KITCHENS Action Detection Challenge at CVPR 2026

- 分离建模动词和名词时序特征,用因果建模提升时间定位精度
- 在100个动作类别上达到82.3% mAP,优于基线模型
- 适合需要高精度时间定位的第一人称视频分析任务
EPIC-KITCHENS-100 动作检测挑战要求模型在长而未剪辑的第一人称视频中定位每个动作的起止时间并标注对应动词-名词标签。本文提出 EgoAction 系统,采用解耦式检测与融合流程:使用 EPIC-finetuned VideoMAE-L 特征,分别训练动词与名词的时序检测器,利用因果建模捕捉时间依赖;通过筛选前 K 个动词-名词组合生成动作假设,并引入置信度自适应边界融合规则。关键观察发现,动词分数对运动变化敏感,名词分数则受手物可见性与物体杂乱影响,固定均值融合会放大单一流程失效时的定位误差。因此提出动态加权融合(DWF),将动词与名词最高分类置信度归一化为提案级权重,线性融合两段边界。该轻量级张量操作仅需替换原有融合策略,即可根据可靠性动态转移边界判定权,同时保持解耦打分机制。结合滑动窗口推理、前K名组合及类别自适应软非极大值抑制,EgoAction 构建了一个紧凑且可复现的第一人称时序动作检测系统。
原文摘要 · Abstract (English)
The EPIC-KITCHENS-100 Action Detection challenge evaluates whether a model can localize the start and end of each action in long untrimmed egocentric videos and assign the corresponding verb--noun action label. In this report, we formulate our submission as EgoAction (Egocentric Action Composition with Reliability-Aware Temporal Fusion), a unified decoupled detection and fusion pipeline. The pipeline uses EPIC-finetuned VideoMAE-L features, trains separate noun and verb temporal detectors with causal temporal modeling, composes action hypotheses from top noun--verb pairs, and introduces a confidence-adaptive boundary fusion rule at post-processing time. The key observation is that verb and noun streams often fail differently: verb scores are sensitive to motion transitions, whereas noun scores are sensitive to hand-object visibility and object clutter. A fixed arithmetic mean of their predicted boundaries can therefore amplify localization errors when one stream degenerates. We replace this hard-coded mean with Dynamic Weighted Fusion (DWF), which normalizes the maximum noun and verb classification confidences into proposal-wise boundary weights and linearly combines the two intervals. This lightweight tensor-only operator shifts boundary authority toward the more reliable stream while preserving the decoupled action scoring mechanism. Together with sliding-window inference, top-K noun--verb action composition, and class-wise Soft-NMS, EgoAction provides a compact and reproducible system for egocentric temporal action detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。