用李群差分偏置提升视频去雨效率与精度
DeLiVR: Differential Spatiotemporal Lie Bias for Efficient Video Deraining
- 引入李群差分偏置,直接作用于注意力分数实现时空一致对齐
- 旋转约束偏置与帧间角速度估计协同,精准匹配雨丝方向
- 比传统光流方法更快更鲁棒,适合实时视频去雨任务
野外拍摄的视频常受雨条、模糊和噪声影响,且相机姿态微小变化会加剧帧间不匹配与时间伪影。现有方法依赖光流或启发式对齐,计算量大且鲁棒性差。李群能精确表示连续几何变换,适用于视频建模中的时空一致性约束。为此,我们提出DeLiVR,一种高效的视频去雨方法,将时空李群差分偏置直接注入网络注意力分数中。具体包括两个互补组件:一是旋转受限的李相对偏置,通过紧凑预测模块估计每帧的平面角度,对归一化坐标进行旋转并对比基准坐标,实现特征聚合前的几何一致性对齐;二是差分群位移,计算相邻帧间的角差以估计运动速度,结合时间衰减与注意力掩码,聚焦帧间关系并精确匹配雨条方向。大量实验表明该方法在公开基准上表现优异,代码已开源。
原文摘要 · Abstract (English)
Videos captured in the wild often suffer from rain streaks, blur, and noise. In addition, even slight changes in camera pose can amplify cross-frame mismatches and temporal artifacts. Existing methods rely on optical flow or heuristic alignment, which are computationally expensive and less robust. To address these challenges, Lie groups provide a principled way to represent continuous geometric transformations, making them well-suited for enforcing spatial and temporal consistency in video modeling. Building on this insight, we propose DeLiVR, an efficient video deraining method that injects spatiotemporal Lie-group differential biases directly into attention scores of the network. Specifically, the method introduces two complementary components. First, a rotation-bounded Lie relative bias predicts the in-plane angle of each frame using a compact prediction module, where normalized coordinates are rotated and compared with base coordinates to achieve geometry-consistent alignment before feature aggregation. Second, a differential group displacement computes angular differences between adjacent frames to estimate a velocity. This bias computation combines temporal decay and attention masks to focus on inter-frame relationships while precisely matching the direction of rain streaks. Extensive experimental results demonstrate the effectiveness of our method on publicly available benchmarks. The code is publicly available at https://github.com/Shuning0312/ICLR-DeLiVR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。