提出首个视频镜头眩光合成与去除方法,解决动态眩光与场景运动独立性难题。
Physics-Informed Video Flare Synthesis and Removal Leveraging Motion Independence between Flare and Scene
- 基于光学流模拟光源运动,建模散射与反射眩光的时序特性。
- 在真实与合成数据上均优于现有方法,有效消除动态眩光并保持时空一致性。
- 构建首个视频眩光数据集,支持动态眩光的系统评估,适合视频修复与增强研究者。
镜头眩光由强光源引起,现有研究多集中于图像,视频眩光的时空特性仍待探索。由于眩光、光源与场景内容间存在复杂且独立的运动,视频眩光合成与去除面临更大挑战,常导致闪烁与伪影。为此,本文提出物理信息驱动的动态眩光合成流程,利用光学流模拟光源运动,并建模散射与反射眩光的时序行为。同时设计一种视频眩光去除网络,采用注意力模块抑制眩光区域,并引入基于Mamba的时序建模组件以捕捉长程时空依赖关系。该运动无关的时空表示无需多帧对齐,缓解了眩光与场景间的时序混叠,提升重建效果。在此基础上,构建首个视频眩光数据集,包含大量合成配对视频及从互联网收集的真实视频,用于评估泛化能力。大量实验表明,本方法在真实与合成视频上均显著优于现有视频修复与图像级眩光去除方法,能有效去除动态眩光,保留光源完整性,并维持场景时空一致性。
原文摘要 · Abstract (English)
Lens flare is a degradation phenomenon caused by strong light sources. Existing researches on flare removal have mainly focused on images, while the spatiotemporal characteristics of video flare remain largely unexplored. Video flare synthesis and removal pose significantly greater challenges than in image, owing to the complex and mutually independent motion of flare, light sources, and scene content. This motion independence further affects restoration performance, often resulting in flicker and artifacts. To address this issue, we propose a physics-informed dynamic flare synthesis pipeline, which simulates light source motion using optical flow and models the temporal behaviors of both scattering and reflective flares. Meanwhile, we design a video flare removal network that employs an attention module to spatially suppress flare regions and incorporates a Mamba-based temporal modeling component to capture long range spatio-temporal dependencies. This motion-independent spatiotemporal representation effectively eliminates the need for multi-frame alignment, alleviating temporal aliasing between flares and scene content and thereby improving video flare removal performance. Building upon this, we construct the first video flare dataset to comprehensively evaluate our method, which includes a large set of synthetic paired videos and additional real-world videos collected from the Internet to assess generalization capability. Extensive experiments demonstrate that our method consistently outperforms existing video-based restoration and image-based flare removal methods on both real and synthetic videos, effectively removing dynamic flares while preserving light source integrity and maintaining spatiotemporal consistency of scene.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。