用新型状态空间模型和相位畸变学习,高效缓解远距离成像湍流失真。
Learning Phase Distortion with Selective State Space Models for Video Turbulence Mitigation
- 基于选择性状态空间模型,实现时空全局感知且计算线性增长。
- 在合成与真实数据上均超越现有方法,推理速度显著更快。
- 新相位畸变表示法更贴近真实湍流效应,提升退化估计能力。
大气湍流是长距离成像系统中图像退化的主因。尽管已有众多深度学习湍流缓解(TM)方法,但多数存在速度慢、内存占用高且泛化能力差的问题。空间域的卷积方法受感受野限制,难以处理湍流所需的大幅空间依赖;时间域的自注意力机制虽理论上可利用湍流的幸运效应,但其二次复杂度难以扩展至多帧。传统递归聚合方法也面临并行化难题。本文提出一种新方法:(1)基于选择性状态空间模型(MambaTM)的湍流缓解网络,每层在时空维度上具备全局感受野,同时保持线性计算复杂度;(2)学习的潜在相位畸变(LPD)引导状态空间模型。不同于传统的泽尼克基表示,新LPD图能唯一捕捉湍流的真实影响,显著降低问题病态性。所提方法在多种合成与真实世界基准上超越现有最先进网络,且推理速度大幅领先。
原文摘要 · Abstract (English)
Atmospheric turbulence is a major source of image degradation in long-range imaging systems. Although numerous deep learning-based turbulence mitigation (TM) methods have been proposed, many are slow, memory-hungry, and do not generalize well. In the spatial domain, methods based on convolutional operators have a limited receptive field, so they cannot handle a large spatial dependency required by turbulence. In the temporal domain, methods relying on self-attention can, in theory, leverage the lucky effects of turbulence, but their quadratic complexity makes it difficult to scale to many frames. Traditional recurrent aggregation methods face parallelization challenges. In this paper, we present a new TM method based on two concepts: (1) A turbulence mitigation network based on the Selective State Space Model (MambaTM). MambaTM provides a global receptive field in each layer across spatial and temporal dimensions while maintaining linear computational complexity. (2) Learned Latent Phase Distortion (LPD). LPD guides the state space model. Unlike classical Zernike-based representations of phase distortion, the new LPD map uniquely captures the actual effects of turbulence, significantly improving the model's capability to estimate degradation by reducing the ill-posedness. Our proposed method exceeds current state-of-the-art networks on various synthetic and real-world TM benchmarks with significantly faster inference speed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。