轻量级递归模型,实时修复大气湍流导致的视频模糊与抖动
RMFAT: Recurrent Multi-scale Feature Atmospheric Turbulence Mitigator
- 用双帧输入递归处理,大幅降低计算负担
- 比现有方法提升近9%的清晰度(SSIM),速度提升超4倍
- 适合嵌入式设备等资源受限场景的实时视频修复
大气湍流会引入几何扭曲、模糊和时间闪烁等失真,严重破坏视频画质,影响视觉清晰度和时序一致性。现有先进方法基于Transformer和3D架构,需多帧输入,但计算成本高、内存占用大,难以实现实时部署,尤其在资源受限场景下。本文提出RMFAT:一种轻量级递归多尺度特征大气湍流抑制模型,旨在实现高效且时序一致的视频恢复。该模型采用仅需两帧输入的递归框架,显著缩小时间窗口并减轻计算压力;同时在编码器和解码器阶段集成多尺度特征编码解码与时间形变模块,增强空间细节和时序连贯性。在合成与真实大气湍流数据集上的大量实验表明,RMFAT不仅在清晰度恢复上优于现有方法(SSIM提升近9%),且推理速度显著加快(运行时间减少超过四倍),特别适用于实时大气湍流抑制任务。
原文摘要 · Abstract (English)
Atmospheric turbulence severely degrades video quality by introducing distortions such as geometric warping, blur, and temporal flickering, posing significant challenges to both visual clarity and temporal consistency. Current state-of-the-art methods are based on transformer and 3D architectures and require multi-frame input, but their large computational cost and memory usage limit real-time deployment, especially in resource-constrained scenarios. In this work, we propose RMFAT: Recurrent Multi-scale Feature Atmospheric Turbulence Mitigator, designed for efficient and temporally consistent video restoration under AT conditions. RMFAT adopts a lightweight recurrent framework that restores each frame using only two inputs at a time, significantly reducing temporal window size and computational burden. It further integrates multi-scale feature encoding and decoding with temporal warping modules at both encoder and decoder stages to enhance spatial detail and temporal coherence. Extensive experiments on synthetic and real-world atmospheric turbulence datasets demonstrate that RMFAT not only outperforms existing methods in terms of clarity restoration (with nearly a 9\% improvement in SSIM) but also achieves significantly improved inference speed (more than a fourfold reduction in runtime), making it particularly suitable for real-time atmospheric turbulence suppression tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。