arXiv:2511.18174cs.CV2025-11中稿 · CVPR被引 1

将任意镜头图像无失真映射到球面,实现旋转等变的高效感知。

Unified Spherical Frontend: Learning Rotation-Equivariant Representations of Spherical Images from Any Camera

  • 通过光线方向映射将任意校准镜头图像转至单位球面,空间域操作保持几何一致性。
  • 在多种数据集上实现接近无损性能,随机测试旋转下性能下降小于1%。
  • 无需旋转增强训练即可零样本适配新镜头,适合广角与全景视觉任务。

现代感知越来越多依赖鱼眼、全景等大视场(FoV)相机,但多数流程仍使用为针孔成像设计的平面卷积神经网络处理二维网格,导致像素邻域无法真实反映物理相邻性,且对全局旋转敏感。传统球面卷积神经网络虽部分缓解此问题,但需昂贵的球谐变换,限制分辨率与效率。我们提出统一球面前端(USF),一种无失真、镜头无关的框架,通过光线方向对应关系将任意校准镜头图像映射至单位球面,并在空间域中进行球面重采样、卷积与池化。USF 模块化设计:投影、位置采样、值插值与分辨率控制完全解耦;其可配置的距离仅卷积核实现旋转等变性,类比平面CNN的平移等变性,且完全避免球谐变换。我们在合成数据集(Spherical MNIST)和真实数据集(PANDORA、Stanford 2D-3D-S)上对比多个标准平面主干与对应的球面版本,在分类、检测与分割任务中评估,同时测试极端镜头畸变、视场变化及任意旋转下的鲁棒性。USF 可高效扩展至高分辨率球面图像,在随机测试时旋转下性能损失低于1%,且无需训练时旋转增强即可实现对任意未见(宽视场)镜头的零样本泛化,性能下降极小。

原文摘要 · Abstract (English)

Modern perception increasingly relies on fisheye, panoramic, and other wide field-of-view (FoV) cameras, yet most pipelines still apply planar CNNs designed for pinhole imagery on 2D grids, where pixel-space neighborhoods misrepresent physical adjacency and models are sensitive to global rotations. Traditional spherical CNNs partially address this mismatch but require costly spherical harmonic transform that constrains resolution and efficiency. We present Unified Spherical Frontend (USF), a distortion-free lens-agnostic framework that transforms images from any calibrated camera onto the unit sphere via ray-direction correspondences, and performs spherical resampling, convolution, and pooling canonically in the spatial domain. USF is modular: projection, location sampling, value interpolation, and resolution control are fully decoupled. Its configurable distance-only convolution kernels offer rotation-equivariance, mirroring translation-equivariance in planar CNNs while avoiding harmonic transforms entirely. We compare multiple standard planar backbones with their spherical counterparts across classification, detection, and segmentation tasks on synthetic (Spherical MNIST) and real-world (PANDORA, Stanford 2D-3D-S) datasets, and stress-test robustness to extreme lens distortions, varying FoV, and arbitrary rotations. USF scales efficiently to high-resolution spherical imagery and maintains less than 1% performance drop under random test-time rotations without training-time rotational augmentation, and enables zero-shot generalization to any unseen (wide-FoV) lenses with minimal performance degradation.

球面表示旋转等变全景视觉镜头无关

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。