解决手机双摄异构导致的不对称模糊问题,提升XR内容画质。
A Benchmark for Heterogeneous Stereo Deblurring with Physically- and Epipolar-constrained Cross Attention

- 提出物理与极线约束的跨视图注意力机制,精准匹配左右图像特征。
- 在真实手机采集数据集上,显著提升去模糊效果且计算开销低。
- 适用于各类网络结构,特别适合移动端视觉应用开发者。
现代支持立体拍摄的智能手机可捕捉沉浸式XR内容,但不同摄像头模块间的硬件差异常引发严重的非对称模糊伪影。现有方法和基准大多假设双摄配置一致,未明确处理此类非对称退化问题。为此,我们提出专门针对异构立体去模糊的框架:首先构建了基于真实手机立体拍摄、通过多帧融合生成的异构立体去模糊(HSD)数据集;其次提出物理与极线约束的交叉注意力(PECA)模块,将跨视图匹配限制在由光学推导出的最大视差边界内的极线搜索窗口内。通过施加物理合理的视差约束,PECA实现高效可靠的跨视图特征融合。此外,结合置信度加权注意力与残差融合机制,在对应关系可靠时强化跨视图去模糊,而在遮挡或不可靠区域则自然退化为自去模糊。PECA具有架构无关性,能持续提升基于CNN、Transformer及NAFNet的基线模型性能。在HSD数据集上的大量实验表明,引入PECA的模型在保持良好效率的同时,实现了更优的复原效果。
原文摘要 · Abstract (English)
Modern stereo-capable smartphones enable immersive XR content capture. However, hardware heterogeneity across camera modules often causes severe asymmetric blur artifacts. Existing methods and benchmarks largely assume homogeneous stereo setups and therefore do not explicitly address such asymmetric degradation. To bridge this gap, we present a dedicated framework for heterogeneous stereo deblurring. First, we introduce the heterogeneous stereo deblurring (HSD) dataset, constructed from real smartphone stereo captures via multi-frame integration. Second, we propose physically- and epipolar-constrained cross attention (PECA), a lightweight module that restricts cross-view matching to an epipolar search window bounded by a optics-derived disparity upper bound. By enforcing physically valid disparity constraints, PECA enables efficient and reliable cross-view feature fusion. Moreover, our confidence-weighted attention with residual fusion emphasizes cross-guided deblurring when correspondences are reliable, while naturally falling back to self-deblurring in occluded or unreliable regions. PECA is architecture-agnostic and consistently improves CNN-, Transformer-, and NAFNet-based baselines. Extensive experiments on HSD show that PECA-enhanced models achieve improved restoration performance with favorable efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。