通过自适应退出机制，提前检测流程视频中的操作错误并减少观看时长。
MistExit: Learning to Exit for Early Mistake Detection in Procedural Videos
- 基于检测器与强化学习策略的双模块设计，实时评估关键步骤正确性。
- 在多个真实数据集上实现更高准确率，同时仅需观察更少视频内容。
- 适合需要低延迟、高效率的自动化质检场景，如工业流水线监控。
我们提出早期错误检测任务：在观看最少视频流的情况下，判断流程性活动中的关键步骤是否执行正确。为此，我们设计了包含错误检测器与强化学习策略的方法。每一步时间，检测器分析近期帧并预测未来视觉特征，以实现可靠的早期错误判断；同时，策略模块融合检测结果与视觉信息，动态决定何时退出（停止处理新帧）并输出最终预测。在多个真实世界流程视频数据集上的实验表明，MistExit模型在保持更高错误检测精度的同时，显著减少了所需观看的视频比例，优于当前最优方法。
原文摘要 · Abstract (English)
We introduce the task of early mistake detection in video, where the goal is to determine whether a keystep in a procedural activity is performed correctly while observing as little of the streaming video as possible. To tackle this problem, we propose a method comprising a mistake detector and a reinforcement learning policy. At each timestep, the detector processes recently observed frames to estimate the keystep's correctness while anticipating future visual features, enabling reliable early mistake estimates. Meanwhile, the policy aggregates the detector outputs and visual observations over time and adaptively decides when to exit (i.e., stop processing incoming frames) while producing the final prediction. Using diverse real-world procedural video datasets, we demonstrate that our MistExit model achieves superior mistake detection accuracy while reducing the fraction of video observed compared to state-of-the-art models. Project: https://vision.cs.utexas.edu/projects/mist_exit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。