arXiv:2604.18747cs.CV2026-04中稿 · ECCV被引 4

提出跨视角/维度的通用位置编码,让Transformer更好处理3D视觉任务。

URoPE: Universal Relative Position Embedding across Geometric Spaces

  • 基于相机射线采样3D点并投影到查询图平面,实现跨空间位置编码
  • 在多类视觉任务中提升性能,如新视角合成和3D检测,均取得一致改进
  • 无需额外参数,兼容现有优化算子,适合需要几何推理的模型

相对位置编码已成为Transformer中编码位置信息的标准机制。然而,现有方法通常局限于固定几何空间,如一维序列或规则二维/三维网格,限制了其在需跨视角或跨维度几何推理的计算机视觉任务中的应用。为此,本文提出URoPE,一种将旋转位置编码(RoPE)扩展至跨视角或跨维度几何空间的通用方法。对于每个键/值图像块,URoPE沿对应相机射线在预设深度锚点处采样3D点,并将其投影到查询图像平面,随后使用投影后的像素坐标应用标准2D RoPE。URoPE是无参数、感知相机内参的相对位置编码,对全局坐标系选择不变,且完全兼容现有RoPE优化注意力核。我们在多种任务中评估了作为插件式位置编码的效果,涵盖新视角合成、3D目标检测、目标跟踪与深度估计,涉及2D-2D、2D-3D及时间场景。实验表明,URoPE在所有任务中均持续提升基于Transformer模型的性能,证明其在几何推理中的有效性与通用性。代码已公开于项目网站:https://urope-pe.github.io/。

原文摘要 · Abstract (English)

Relative position embedding has become a standard mechanism for encoding positional information in Transformers. However, existing formulations are typically limited to a fixed geometric space, namely 1D sequences or regular 2D/3D grids, which restricts their applicability to many computer vision tasks that require geometric reasoning across camera views or between 2D and 3D spaces. To address this limitation, we propose URoPE, a universal extension of Rotary Position Embedding (RoPE) to cross-view or cross-dimensional geometric spaces. For each key/value image patch, URoPE samples 3D points along the corresponding camera ray at predefined depth anchors and projects them into the query image plane. Standard 2D RoPE can then be applied using the projected pixel coordinates. URoPE is a parameter-free and intrinsics-aware relative position embedding that is invariant to the choice of global coordinate systems, while remaining fully compatible with existing RoPE-optimized attention kernels. We evaluate URoPE as a plug-in positional encoding for transformer architectures across a diverse set of tasks, including novel view synthesis, 3D object detection, object tracking, and depth estimation, covering 2D-2D, 2D-3D, and temporal scenarios. Experiments show that URoPE consistently improves the performance of transformer-based models across all tasks, demonstrating its effectiveness and generality for geometric reasoning. Our code is available on our project website: https://urope-pe.github.io/.

位置编码3D视觉Transformer几何推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。