融合可见光与热成像，提升复杂环境下路面裂缝分割精度
IRFusionFormer: Enhancing Pavement Crack Segmentation with RGB-T Fusion and Topological-Based Loss
- 设计跨模态融合模块，有效捕捉多尺度与长距离特征关联
- 提出拓扑损失函数，保持裂缝结构连通性，提升分割连续性
- 在低光照等恶劣条件下表现优异，适合实际工程部署
路面裂缝分割对评估道路结构完整性至关重要。尽管深度学习在可见光图像分割上取得进展，但在弱光或运动模糊等恶劣条件下性能下降。热成像可捕捉辐射信息，提供互补数据，改善复杂环境下的裂缝检测。结合可见光与热成像（RGB-T）进行分割虽具潜力，但现有方法未能充分挖掘双模态间的多层次互补关系。为此，我们提出IRFusionFormer，一种新型裂缝分割模型。其高效的跨模态融合模块在不显著增加计算开销的前提下，捕获双模态间的多尺度关系与长程依赖。同时，引入交互-混合-分支-监督框架，通过分支间联合监督增强模态交互。为保持裂缝拓扑结构，提出新型拓扑损失函数，在训练中保留裂缝连通性。实验表明，该方法在标准数据集上达到90.01%的Dice分数和81.83%的IoU，显著提升不同环境下的鲁棒性与准确性。相关代码、数据与模型已公开于https://github.com/sheauhuu/IRFusionFormer。
原文摘要 · Abstract (English)
Crack segmentation is crucial in civil engineering, particularly for assessing pavement integrity and ensuring the durability of infrastructure. While deep learning has advanced RGB-based segmentation, performance degrades under adverse conditions like low illumination or motion blur. Thermal imaging offers complementary information by capturing emitted radiation, improving crack detection in challenging environments. Combining RGB and thermal images (RGB-T) for crack segmentation shows promise in complex real-world conditions, such as adverse weather, yet research in this area remains limited. Current RGB-T segmentation methods often fail to fully exploit the complementary relationships between modalities at various levels of interaction. To address this, we propose IRFusionFormer, a novel model for crack segmentation that effectively integrates RGB and thermal data. Our Efficient RGB-T Cross Fusion Module captures multi-scale relationships and long-range dependencies between modalities without significant computational overhead. Additionally, we introduce the Interaction-Hybrid-Branch-Supervision framework, which enhances interaction between modalities by distributing fused features across branches with joint supervision. To maintain the topological structure of cracks, we introduce a novel topology-based loss function that preserves connectivity during training. Our method achieves state-of-the-art performance, with a Dice score of 90.01% and an IoU of 81.83%, significantly improving robustness and accuracy in varying environmental conditions. These advancements address key challenges in pavement crack segmentation, offering a more reliable and efficient solution. For access to the codes, data, and models from this study, visit https://github.com/sheauhuu/IRFusionFormer
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。