基于冻结特征的双阶段融合模型，提升厨房场景动作预测准确率。
TAP-JEPA: Frozen Future-Latent Probing and Two-Stage Score Fusion for EPIC-KITCHENS-100 Action Anticipation
- 用冻结的V-JEPA特征+注意力探针，融合可见动作与未来预测特征
- 在EPIC-KITCHENS-100上达到27.91%的总体Top-5召回率，排名第二
- 适合关注自监督视频表征与动作前瞻任务的研究者
本文介绍TAP-JEPA，即我们在EgoVis 2026会议上针对EPIC-KITCHENS-100（EK-100）动作前瞻挑战赛的参赛方案。任务要求从一个尚未开始目标动作的视角视频片段中，预测下一个动词、名词及动词-名词组合。我们未微调大型视频主干网络，而是在冻结的V-JEPA 2.1特征基础上构建轻量级前瞻模型：使用ViT-G/384编码器提取可见预动作标记，预训练的潜在预测器基于观察上下文估计近未来标记，再通过任务特定查询的注意力探针融合两组标记。最终提交版本扩大了监督训练范围，涵盖官方训练集及大部分验证集，仅保留小部分用于校验与定性分析，并采用两阶段分数融合策略：先对每轮中8个独立初始化的探针副本取平均，再将第12至20轮的候选结果按领域依赖权重合并。在官方公开测试排行榜上，我们的sunshinesky条目获得27.91%的整体动作平均Top-5召回率（MT5R），位列第二，仅落后于第一名0.04个百分点。
原文摘要 · Abstract (English)
This report presents TAP-JEPA, our runner-up submission to the EPIC-KITCHENS-100 (EK-100) Action Anticipation Challenge at EgoVis 2026. The task is to anticipate the next verb, noun, and verb-noun action from an egocentric clip that ends before the target action begins. Instead of fine-tuning a large video backbone, TAP-JEPA builds a compact anticipation model on frozen V-JEPA 2.1 features: a ViT-G/384 encoder extracts visible pre-action tokens, the pre-trained latent predictor estimates near-future tokens from the observed context, and both token groups are fused by attentive probes with task-specific queries for verbs, nouns, and action pairs. For the final submission, we expand supervised training with the official training split and most of the validation split, reserving a small subset for sanity checks and qualitative inspection, and adopt a two-stage score fusion that first averages eight independently initialized probe replicas within each epoch and then merges candidates from epochs 12-20 with field-dependent weights. On the official open-testing leaderboard, our sunshinesky entry achieves 27.91 percent overall action Mean Top-5 Recall (MT5R), ranking second and only 0.04 percentage points behind the top score.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。