arXiv:2606.25317cs.CVcs.AI2026-06

利用动作检测器预测不一致性,实现轻量级实时流程错误检测

ESTANet: Efficient Online Error Detection in Procedural Videos via Prediction Inconsistency

论文配图:ESTANet: Efficient Online Error Detection in Procedural Videos via Prediction Inconsistency
图 1 · 摘自论文原文
  • 通过标准与敏感型检测器的预测差异识别错误
  • 在三个数据集上达到领先性能,推理延迟低于50ms
  • 无需复杂结构或额外标注,适合日常场景应用

实时在线流程错误检测对日常生活支持至关重要,可即时提示并引导纠错。本文从动作检测器自身预测行为这一被忽视的角度出发,观察到检测器对输入动态和时序上下文的敏感度不同,导致正常执行下预测一致,出错时则出现差异。为此提出ESTANet(Error-Sensitive and Temporally-vArying Network),一个轻量级框架,通过少量动作检测器间的预测不一致性来检测错误。构建标准与误差敏感型检测器,在正确执行时表现相似,出错时响应不同;结合不同时间上下文的检测器进一步放大偏差。推理时通过多数投票聚合标准与敏感预测间的不匹配,标记含错帧。在EgoPER、Assembly-101-O和EPIC-Tent-O上实验表明,ESTANet实现最佳性能,同时保持轻量化与实时性(<50ms延迟)。结果证明,利用检测器内在特性即可实现高效实用的在线错误检测,无需增加架构复杂度。

原文摘要 · Abstract (English)

An efficient and accurate system for detecting errors in procedural tasks is crucial for supporting human needs in daily life, as it can provide instant notifications and guide people to correct mistakes. In this work, we study real-time online error detection in procedural videos from a simple but overlooked perspective: the prediction behavior of action detectors themselves. Instead of designing complex architectures or specialized supervision, we observe that action detectors naturally exhibit different prediction characteristics depending on their sensitivity to input dynamics and temporal context. We therefore propose ESTANet (Error-Sensitive and Temporally-vArying Network), a lightweight framework that detects errors by exploiting inconsistencies among action predictions produced by a small set of action detectors. We construct standard and error-sensitive action detectors that behave similarly on correct executions but respond differently when errors occur. Meanwhile, detectors operating with different temporal contexts further amplify prediction inconsistencies when the procedure deviates from the intended sequence. During inference, we detect errors by aggregating mismatches between standard and error-sensitive predictions through majority voting to flag frames that contain errors. Extensive experiments on EgoPER, Assembly-101-O, and EPIC-Tent-O demonstrate that ESTANet achieves state-of-the-art performance in online error detection while maintaining real-time efficiency with a lightweight architecture. Our results highlight that leveraging the intrinsic properties of action detectors can yield a powerful and practical solution for online error detection without increasing architectural design complexity.

视频分析错误检测轻量模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。