arXiv:2510.03666cs.CVcs.AI2025-10被引 7

用视觉语言模型自动识别矿场违规行为,提升安全监控效率。

MonitorVLM:A Vision Language Framework for Safety Violation Detection in Mining Operations

  • 构建矿场违规多模态数据集,含9000个问答样本。
  • 引入条款筛选与行为增强模块,精度提升3.45%,召回率提高8.62%。
  • 轻量级网页界面支持实时报警与视频时间戳,适合工程落地。

工业事故在露天和地下采矿等高危领域常由工人不安全行为引发。传统人工巡检成本高、易出错,难以适应大规模动态环境,亟需智能自动化监控方案。本文提出MonitorVLM,一种直接从监控视频流中检测安全违规的视觉-语言框架。其核心创新包括:(1) 构建包含9000个视觉-问题-答案样本的领域专用违规数据集,覆盖40项高频矿场规范,并通过增强与辅助检测线索丰富;(2) 提出条款过滤(CF)模块,动态选择最相关前K条条款,推理延迟降低13.56%且保持精度;(3) 设计行为放大(BM)模块,增强工人区域以提升细粒度动作识别,使精度额外提升3.45%,召回率提高8.62%。实验表明,MonitorVLM显著优于基线视觉-语言模型,在精度、召回率和F1分数上分别比72B未微调模型提升22.01%、34.22%和28.37%。配套轻量级网页界面实现自动违规报告与视频时间戳功能,便于实际部署。本研究展示了多模态大模型在矿场乃至更广泛场景下职业安全监控中的潜力。

原文摘要 · Abstract (English)

Industrial accidents, particularly in high-risk domains such as surface and underground mining, are frequently caused by unsafe worker behaviors. Traditional manual inspection remains labor-intensive, error-prone, and insufficient for large-scale, dynamic environments, highlighting the urgent need for intelligent and automated safety monitoring. In this paper, we present MonitorVLM, a novel vision--language framework designed to detect safety violations directly from surveillance video streams. MonitorVLM introduces three key innovations: (1) a domain-specific violation dataset comprising 9,000 vision--question--answer (VQA) samples across 40 high-frequency mining regulations, enriched with augmentation and auxiliary detection cues; (2) a clause filter (CF) module that dynamically selects the Top-$K$ most relevant clauses, reducing inference latency by 13.56\% while maintaining accuracy; and (3) a behavior magnifier (BM) module that enhances worker regions to improve fine-grained action recognition, yielding additional gains of 3.45% in precision and 8.62% in recall. Experimental results demonstrate that MonitorVLM significantly outperforms baseline vision--language models, achieving improvements of 22.01% in precision, 34.22\% in recall, and 28.37% in F1 score over the 72B unfine-tuned baseline. A lightweight web-based interface further integrates MonitorVLM into practical workflows, enabling automatic violation reporting with video timestamping. This study highlights the potential of multimodal large models to enhance occupational safety monitoring in mining and beyond.

安全监控视觉语言模型矿业违规检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。