arXiv:2504.09083cs.CV2025-04被引 15

用视觉语言模型识别工地隐患,理解上下文关系更准。

Using Vision Language Models for Safety Hazard Identification in Construction

  • 将安全规范转为提示词,让模型理解场景上下文
  • GPT-4o和Gemini在1100张图上达到0.906的BERTScore
  • 适合想提升工地安全预警能力的工程团队

安全风险识别与预防是主动安全管理的关键。以往研究多利用计算机视觉从工地图像中自动识别隐患,但这些方法仅关注预定义物体检测,缺乏对空间关系和交互的理解,难以适应不同工地规范和环境,泛化能力差。为此,我们提出并验证了一种基于视觉语言模型(VLM)的工地隐患识别框架。该框架通过提示工程模块将安全规范转化为上下文查询,使VLM能结合视觉信息生成符合规范的隐患判断。我们在包含1100张工地图像的自建数据集上评估了GPT-4o、Gemini、Llama 3.2和InternVL2等主流VLM,结果表明GPT-4o和Gemini 1.5 Pro表现最优,分别获得0.906和0.888的BERTScore,有效识别通用与情境特定隐患。但处理时间仍制约实时应用。研究为VLM在工地安全监测中的落地提供了参考,助力主动安全管理提升。

原文摘要 · Abstract (English)

Safety hazard identification and prevention are the key elements of proactive safety management. Previous research has extensively explored the applications of computer vision to automatically identify hazards from image clips collected from construction sites. However, these methods struggle to identify context-specific hazards, as they focus on detecting predefined individual entities without understanding their spatial relationships and interactions. Furthermore, their limited adaptability to varying construction site guidelines and conditions hinders their generalization across different projects. These limitations reduce their ability to assess hazards in complex construction environments and adaptability to unseen risks, leading to potential safety gaps. To address these challenges, we proposed and experimentally validated a Vision Language Model (VLM)-based framework for the identification of construction hazards. The framework incorporates a prompt engineering module that structures safety guidelines into contextual queries, allowing VLM to process visual information and generate hazard assessments aligned with the regulation guide. Within this framework, we evaluated state-of-the-art VLMs, including GPT-4o, Gemini, Llama 3.2, and InternVL2, using a custom dataset of 1100 construction site images. Experimental results show that GPT-4o and Gemini 1.5 Pro outperformed alternatives and displayed promising BERTScore of 0.906 and 0.888 respectively, highlighting their ability to identify both general and context-specific hazards. However, processing times remain a significant challenge, impacting real-time feasibility. These findings offer insights into the practical deployment of VLMs for construction site hazard detection, thereby contributing to the enhancement of proactive safety management.

视觉语言模型工地安全隐患识别

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。