提出快速全景图像超分模型FAOR,利用球面几何先验实现高效任意尺度重建。
Fast Omni-Directional Image Super-Resolution: Adapting the Implicit Image Function with Pixel and Semantic-Wise Spherical Geometric Priors
- 通过像素与语义级球面畸变映射,在特征表示阶段引入球面特性。
- 采用基于测地线的重采样策略,重建阶段无需额外参数即可对齐球面几何。
- 相比现有方法速度更快,适合实时全景图像超分任务。
在全景图像超分辨率(ODI-SR)中,等距柱状投影(ERP)导致非均匀过采样,带来独特挑战。现有方法多依赖复杂的球面卷积或多面体重投影,虽提升性能但计算开销大、推理慢。本文提出一种名为FAOR的新模型,可实现快速且任意尺度的ODI-SR。核心创新在于将平面图像的隐式函数适配至ERP域,通过在潜在表示与图像重建阶段引入球面几何先验,以低开销方式实现。具体而言,在潜在表示阶段,采用像素级和语义级球面到平面的畸变映射,对特征进行仿射变换,融入球面属性;在重建阶段,提出基于测地线的重采样策略,使隐式图像函数自然对齐球面几何,无需增加参数。实验结果表明,FAOR在多个基准上超越当前最优模型,同时推理速度显著提升。
原文摘要 · Abstract (English)
In the context of Omni-Directional Image (ODI) Super-Resolution (SR), the unique challenge arises from the non-uniform oversampling characteristics caused by EquiRectangular Projection (ERP). Considerable efforts in designing complex spherical convolutions or polyhedron reprojection offer significant performance improvements but at the expense of cumbersome processing procedures and slower inference speeds. Under these circumstances, this paper proposes a new ODI-SR model characterized by its capacity to perform Fast and Arbitrary-scale ODI-SR processes, denoted as FAOR. The key innovation lies in adapting the implicit image function from the planar image domain to the ERP image domain by incorporating spherical geometric priors at both the latent representation and image reconstruction stages, in a low-overhead manner. Specifically, at the latent representation stage, we adopt a pair of pixel-wise and semantic-wise sphere-to-planar distortion maps to perform affine transformations on the latent representation, thereby incorporating it with spherical properties. Moreover, during the image reconstruction stage, we introduce a geodesic-based resampling strategy, aligning the implicit image function with spherical geometrics without introducing additional parameters. As a result, the proposed FAOR outperforms the state-of-the-art ODI-SR models with a much faster inference speed. Extensive experimental results and ablation studies have demonstrated the effectiveness of our design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。