同时修复湍流模糊并提升目标检测，端到端训练实现双效果
DMAT: An End-to-End Framework for Joint Atmospheric Turbulence Mitigation and Object Detection
- 用3D Mamba结构处理时空扭曲与模糊
- 在湍流数据集上检测准确率提升15%
- 适合无人机巡检、遥感等低质图像场景
大气湍流会降低监控图像的清晰度和准确性，不仅影响视觉质量，还干扰目标分类与场景追踪。现有深度学习方法虽能改善视觉质量，但时空失真问题依然存在。尽管深度学习目标检测在正常条件下表现良好，但在受湍流影响的序列上仍难以有效工作。本文提出一种新框架，可同步补偿失真特征，提升可视化质量与目标检测性能。该端到端训练策略使湍流消除模块与目标检测模块共享低层失真特征与高层语义特征。具体而言，湍流消除模块采用基于3D Mamba的结构，以处理湍流引起的时空位移与模糊；通过在两个模块间反向传播优化。所提DMAT框架在生成湍流数据集上，相比最先进方法在检测性能上最高提升15%。
原文摘要 · Abstract (English)
Atmospheric Turbulence (AT) degrades the clarity and accuracy of surveillance imagery, posing challenges not only for visualization quality but also for object classification and scene tracking. Deep learning-based methods have been proposed to improve visual quality, but spatio-temporal distortions remain a significant issue. Although deep learning-based object detection performs well under normal conditions, it struggles to operate effectively on sequences distorted by atmospheric turbulence. In this paper, we propose a novel framework that learns to compensate for distorted features while simultaneously improving visualization and object detection. This end-to-end training strategy leverages and exchanges knowledge of low-level distorted features in the AT mitigator with semantic features extracted in the object detector. Specifically, in the AT mitigator a 3D Mamba-based structure is used to handle the spatio-temporal displacements and blurring caused by turbulence. Optimization is achieved through back-propagation in both the AT mitigator and object detector. Our proposed DMAT outperforms state-of-the-art AT mitigation and object detection systems up to a 15% improvement on datasets corrupted by generated turbulence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。