用AI自动分析桥梁损伤并打分，减少人工差异。
Fine-Tuning Vision-Language Models for Understanding Current Damage and Scoring Priority with Quality Guard Agent
- 微调视觉语言模型识别损伤部位和类型
- 2000张图训练后效果接近最优，4000张反致性能下降
- 引入质量过滤器，避免错误图像影响评分结果
日本桥梁每五年需强制进行视觉检查，但不同工程师给出的损伤等级（a-e）存在显著主观差异，且资深工程师老龄化加剧了检查能力危机。本文提出一种基于微调视觉语言模型（VLMs）的自动化桥梁损伤理解与维修优先级评分方法。我们使用QLoRA对LLaVA-1.5-7B进行微调，训练数据最多达4000组配对的桥梁损伤图像与检查文本记录，在800张固定测试图像上评估。模型输出自然语言描述损伤结构与模式，再由规则引擎计算五级维修优先级。渐进式训练实验显示，2000样本时验证损失已接近最优（仅2.9小时训练），后续每样本翻倍，验证损失改善不超过0.2%，呈现明显边际递减。此外，测试集语义相似度在3000样本时达到峰值0.6909，4000样本时降至0.6739，表明中等规模、高质量数据优于更大但噪声更多的数据集。通过torch.compile()与批量处理（batch_size=8）优化推理，单图耗时降至10.06秒，相较基线降低70.2%。本方法有助于提升桥梁检查数据治理水平，减少人为差异，并为专家提供智能辅助判级。同时引入双阶段质量守卫机制，使用微调后的Swallow-8B SLM剔除低质量输出，防止模糊或无法识别图像导致误判。
原文摘要 · Abstract (English)
Bridge inspection in Japan requires mandatory visual assessments every five years, yet qualitative damage ratings (levels a-e) assigned by different engineers exhibit significant inter-rater variability -- a critical barrier to consistent infrastructure management. The aging of skilled engineers further threatens inspection capacity. This paper presents a methodology for automating bridge damage understanding and repair priority scoring using fine-tuned Vision-Language Models (VLMs). We fine-tune LLaVA-1.5-7B with QLoRA on up to 4,000 paired bridge damage images and inspection text records, then evaluate on a fixed test set of 800 images. The model outputs natural language descriptions identifying structural members and damage patterns, from which a rule-based scoring engine calculates a five-level repair priority index. A progressive training study (1k/2k/3k/4k samples) reveals that 2k training samples achieve near-optimal validation loss in only 2.9 hours of training; beyond 2k, validation loss improves by no more than 0.2% per doubling of training samples, exhibiting clear diminishing returns. Furthermore, semantic similarity on the held-out test set peaks at 3k (0.6909) and degrades at 4k (0.6739), indicating that quality-curated mid-scale data outperforms larger but noisier corpora. Inference optimization combining torch.compile() and batch processing (batch_size=8) achieves 10.06 seconds per image -- a 70.2% reduction over the unoptimized baseline. Our approach contributes to data governance in bridge inspection, reduces inter-rater variability, and provides AI-assisted triage to augment expert engineers in inspection workflows. Furthermore, we introduce a two-stage Quality Guard using a fine-tuned Swallow-8B SLM to reject low-quality VLM outputs before priority scoring, preventing spurious scores from damaged or unrecognised images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。