从视频中自动提取可分析的事件日志，解决多模态数据处理难题。
All Eyes on the Workflow: Automated and Efficient Event Discovery from Video Streams
- 用图像嵌入将帧转为向量，通过相似度矩阵做时间分段。
- 少样本分类精准给视频片段打标签，生成带时间戳的事件序列。
- 适合需要从监控视频挖掘流程信息的研究与企业用户。
业务流程管理与流程挖掘等学科依赖记录的事件数据揭示组织流程洞察。然而，多模态数据（如视频）难以直接解析为事件，成为流程分析的障碍。现有方法依赖预定义活动标签词典，无法提供逐帧标注解释，或使用过时的计算机视觉技术。本文提出SnapLog，通过图像嵌入将视频帧转换为特征向量，利用帧间相似度矩阵进行时间分段，再采用广义少样本分类为视频段落分配标签，生成可解释的、带时间戳的帧序列，即事件日志。该日志可直接用于传统流程挖掘技术分析。实验表明，该方法能准确反映视频中的实际流程。
原文摘要 · Abstract (English)
Disciplines such as business process management and process mining aid organizations by discovering insights about processes on the basis of recorded event data. However, an obstacle to process analysis is data multi-modality: for instance, data in video form are not directly interpretable as events. Existing approaches rely on a dictionary of activity label as input, cannot provide frame-by-frame labeling explanations, or rely on superseded computer vision techniques. In this work, we present SnapLog, an approach to extract event data from videos by converting frames to feature vectors using image embeddings and performing temporal segmentation through frame-wise similarity matrices. A generalized few-shot classification is then used to assign labels to the video segments, yielding labeled, timestamped sub-sequences of frames that are interpretable as events. Conventional process mining techniques can be used to analyze the resulting data. We show that our approach produces logs that accurately reflect the process in the videos.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。