arXiv:2510.19292cs.CV2025-10综述被引 2

用视觉识别流程任务中的错误,提升人机协作安全与效率

Vision-Based Mistake Analysis in Procedural Activities: A Review of Advances and Challenges

  • 通过动作识别与行为预测分析执行偏差
  • 可检测顺序错乱、手法错误和时间不准等失误
  • 适合工业质检、康复训练等需高精度流程监控场景

流程性活动中的错误分析是工业自动化、物理康复、教育及人机协作等领域的关键研究方向。本文综述基于视觉的流程任务错误检测与预测方法,聚焦程序性与执行性错误。借助动作识别、行为预测与活动理解等计算机视觉进展,视觉系统可识别任务执行中的偏离,如顺序错误、不当操作或时间偏差。文章探讨了类内差异、视角变化及组合式活动结构带来的挑战,并系统梳理现有数据集、评估指标与先进方法,按程序结构利用、监督层级与学习策略分类。开放问题包括区分可接受变异与真实错误、建模错误传播等,未来方向涵盖神经符号推理与反事实状态建模。本工作旨在建立统一视角,推动该领域在安全、效率与任务表现上的应用潜力。

原文摘要 · Abstract (English)

Mistake analysis in procedural activities is a critical area of research with applications spanning industrial automation, physical rehabilitation, education and human-robot collaboration. This paper reviews vision-based methods for detecting and predicting mistakes in structured tasks, focusing on procedural and executional errors. By leveraging advancements in computer vision, including action recognition, anticipation and activity understanding, vision-based systems can identify deviations in task execution, such as incorrect sequencing, use of improper techniques, or timing errors. We explore the challenges posed by intra-class variability, viewpoint differences and compositional activity structures, which complicate mistake detection. Additionally, we provide a comprehensive overview of existing datasets, evaluation metrics and state-of-the-art methods, categorizing approaches based on their use of procedural structure, supervision levels and learning strategies. Open challenges, such as distinguishing permissible variations from true mistakes and modeling error propagation are discussed alongside future directions, including neuro-symbolic reasoning and counterfactual state modeling. This work aims to establish a unified perspective on vision-based mistake analysis in procedural activities, highlighting its potential to enhance safety, efficiency and task performance across diverse domains.

视觉分析错误检测流程任务人机协作

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。