跨视角模仿错误检测新框架，提升工业质检准确率
SAVA-X: Ego-to-Exo Imitation Error Detection via Scene-Adaptive View Alignment and Bidirectional Cross View Fusion
- 通过自适应采样与双向交叉注意力融合，对齐第一/第三人称视频
- 在EgoMe数据集上AUPRC和均值tIoU均优于现有方法
- 适合工业训练、医疗操作等跨视角动作评估场景
错误检测在工业培训、医疗和装配质量控制中至关重要。现有方法多假设单视角设置，无法处理第三人称（exo）示范与第一人称（ego）模仿的对比场景。本文提出Ego→Exo模仿错误检测任务：给定异步且长度不匹配的ego与exo视频，模型需在ego时间线上定位步骤并判断是否出错。该任务存在跨视角域偏移、时间错位和冗余问题。在统一评测协议下，我们适配了密集视频字幕和时序动作检测的强基线，发现其在跨视角场景表现不佳。为此提出SAVA-X框架，包含（i）视图条件自适应采样，（ii）场景自适应视图嵌入，（iii）双向交叉注意力融合。在EgoMe基准上，SAVA-X持续提升AUPRC与平均tIoU，消融实验验证各模块互补性。代码已开源。
原文摘要 · Abstract (English)
Error detection is crucial in industrial training, healthcare, and assembly quality control. Most existing work assumes a single-view setting and cannot handle the practical case where a third-person (exo) demonstration is used to assess a first-person (ego) imitation. We formalize Ego$\rightarrow$Exo Imitation Error Detection: given asynchronous, length-mismatched ego and exo videos, the model must localize procedural steps on the ego timeline and decide whether each is erroneous. This setting introduces cross-view domain shift, temporal misalignment, and heavy redundancy. Under a unified protocol, we adapt strong baselines from dense video captioning and temporal action detection and show that they struggle in this cross-view regime. We then propose SAVA-X, an Align-Fuse-Detect framework with (i) view-conditioned adaptive sampling, (ii) scene-adaptive view embeddings, and (iii) bidirectional cross-attention fusion. On the EgoMe benchmark, SAVA-X consistently improves AUPRC and mean tIoU over all baselines, and ablations confirm the complementary benefits of its components. Code is available at https://github.com/jack1ee/SAVAX.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。