用视觉模型精检建筑，用大模型判断损毁，少标注也能准
Assessing the Benefits of Combining Advanced Deep Learning Techniques for Post-Disaster Building Damage Assessment from UAV Imagery
- 先用CV模型精准定位建筑，再交由大模型分析损毁程度
- 在两个真实数据集上，损毁分类准确率提升最多达2.1个R²
- 仅需少量标注数据，适合灾后快速评估场景
灾后建筑损毁评估需快速准确，但现有方法依赖大量标注数据，跨区域泛化差，且任务固定。无人机影像虽能提供高分辨率及时视图，但传统计算机视觉模型存在局限。大型视觉语言模型虽具强推理能力，但在细粒度感知任务（如目标检测、精确框生成）表现不足，且需大量数据微调。本文提出一种混合框架，将检测与损毁评估解耦：先由CV模型检测建筑并生成边界框，再将结果输入LVLM进行损毁分类与上下文理解。在RescueNet和FloodNet两个真实基准上验证，该框架在完整、部分损毁、完全破坏建筑的计数上，相比独立基线最高提升2.1个R²，且检测阶段仅需有限标注数据。文章还深入分析失败案例与边缘情况，为实践者提供指导，未来研究方向亦明确。代码与数据已开源：https://github.com/ungquanghuy-kddi/VLM_GDINO.git
原文摘要 · Abstract (English)
Rapid and accurate post-disaster building damage assessment is essential, yet remains a challenging task. Unmanned Aerial Vehicle (UAV) imagery offers a timely and high-resolution view of affected areas, but existing Computer Vision (CV) models often demand large annotated datasets, generalize poorly across geographic regions and their assessment policies, and are confined to the specific tasks they were trained for. Large Vision-Language Models (LVLMs) offer a promising alternative through their strong reasoning and generalization capabilities, but fall short on precise, low-level perception tasks such as object detection and accurate bounding box generation. Furthermore, they often require a substantial amount of data for effective fine-tuning on domain-specific tasks. In this paper, we propose a hybrid framework that decouples detection from damage assessment, combining the precision of CV models with the reasoning power of LVLMs. A CV model first detects buildings and generates bounding boxes on the image that are then passed to an LVLM for damage classification and contextual interpretation. We evaluated our framework on two real-world benchmarks: RescueNet and FloodNet. In particular, the best combination under this framework accurately counts intact, partially damaged and completely destroyed buildings, surpassing isolated baselines by up to 2.1 R^2 points, while requiring only limited annotated data for the detection stage. Beyond reporting aggregate gains, we provide a detailed analysis of failure scenarios and edge cases, offering practical insights for practitioners and concrete directions for future work. Our source code and data are publicly available to the research community via the following repository: https://github.com/ungquanghuy-kddi/VLM_GDINO.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。