提出混合模型提升光场超分辨率,更好保持4D视角一致性
Slope-Guided Mamba and Angular-Refined Transformer for Light Field Super-Resolution

- 用斜率引导的Mamba与角度优化的Transformer融合建模
- 在5个数据集上比之前方法高0.42 dB(PSNR),伪影更少
- 适合做光场图像处理、高维视觉建模的研究者参考
光场超分辨率(LFSR)需要精确建模空间-角度相关性,同时保持固有的4D射线一致性。然而,现有建模范式存在两个固有局限:一是空间与角度维度常被解耦处理,限制早期跨维度交互,导致几何不一致;二是尽管连续序列建模在表示视差结构方面有潜力,但其固定的扫描机制与视差几何本质冲突,制约了几何感知的特征聚合。为此,我们提出一种混合光场超分辨率网络SMART,融合斜率引导的Mamba与角度精炼的Transformer,有效克服上述问题。具体地,引入角度调制的空间模块,通过角度先验强化空间-角度相关性建模;提出流形对齐轨迹模块,使序列建模沿视差结构进行,实现几何一致的特征聚合。在五个基准数据集上的实验表明,SMART达到最先进性能,相比之前方法提升0.42 dB(PSNR),且明显减少伪影。
原文摘要 · Abstract (English)
Light Field Super-Resolution (LFSR) necessitates accurate modeling of spatial-angular correlations while preserving intrinsic 4D ray coherence. However, maintaining such high-dimensional consistency remains challenging, primarily due to two inherent limitations in prevailing modeling paradigms. First, spatial and angular dimensions are often modeled in a decoupled manner, restricting early cross-dimensional interaction and leading to geometric inconsistencies. Moreover, although continuous sequence modeling paradigms show promise in representing epipolar structures, their rigid scanning mechanisms fundamentally conflict with epipolar geometry, limiting geometry-aware feature aggregation. To address these challenges, we propose a hybrid light field super-resolution network, termed SMART, which integrates a Slope-Guided Mamba and an Angular-Refined Transformer to effectively overcome these limitations. Specifically, we introduce an angular-modulated spatial module to bridge the decoupling gap, incorporating angular priors to strengthen spatial-angular correlation modeling. To mitigate the scan-geometry mismatch, we propose a manifold-aligned trajectory module that enables geometry-consistent sequence modeling along epipolar structures. Experiments on five benchmarks demonstrate that SMART achieves state-of-the-art performance, surpassing previous methods by 0.42 dB (PSNR) with significantly reduced artifacts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。