构建真实施工安全评估基准，提升大模型在复杂场景下的可靠性。
SafeBuild-Bench: A Temporal-Robust Construction Safety Benchmark with Graph-Enhanced Data Mining
- 基于图增强选择机制从海量数据中挖掘高信息量样本
- 涵盖3314个任务实例，覆盖多选与自由描述两类安全识别任务
- 适用于评估大模型在真实施工场景下的鲁棒性，适合安全检测研究者
施工安全模型需应对实际部署风险，如工人靠近无护栏脚手架等场景，而不仅限于在精选图像中识别常见物体。然而真实巡检数据冗余、长尾分布，并跨越不同工地和时间跨度。我们提出SafeBuild-Bench，一个基于元数据的基准，用于评估多模态大语言模型在真实时空变化下的施工安全能力。该基准从超过10万条工业图文记录中挖掘，包含3314个任务实例，来自3000多张专家验证图像，覆盖多选隐患识别与自由文本描述。为实现可扩展的专家验证，我们开发了GEMS——一种结合代理模型混淆信号与图结构多样性的多模态筛选管道，能从冗余数据流中选出高价值候选。在公开指令微调数据上，GEMS筛选的子集在小数据预算下仍保持强鲁棒性。在SafeBuild-Bench上，当前多模态大语言模型表现仍不理想，最佳综合得分接近60。相关代码与数据已开源至https://github.com/safebuild/gems。
原文摘要 · Abstract (English)
Construction-safety models must handle concrete deployment risks, such as a worker standing near a scaffold edge without guardrails, rather than only recognize common objects in curated images. Yet real inspection archives are redundant, long-tailed, and collected across changing sites and months. We introduce SafeBuild-Bench, a metadata-driven benchmark for evaluating multimodal large language models on construction safety under realistic temporal and site variation. It is mined from 100K+ industrial image-text records and contains 3,314 task instances from over 3,000 expert-verified images, covering multiple-choice hazard identification and free-form hazard description. To make expert verification scalable, we develop GEMS, a graph-enhanced multimodal selection pipeline that combines a proxy-model confusion signal with graph-based diversity to identify informative candidates from redundant streams. On public instruction-tuning data, GEMS-selected subsets preserve robustness-oriented performance under small data budgets. On SafeBuild-Bench, current MLLMs remain far from reliable construction-safety understanding, with the best overall score near 60. We release the benchmark, evaluation scripts, and GEMS codebase at https://github.com/safebuild/gems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。