首个支持任意方向的3D形状文本检索模型,解决旋转不变性难题。
RI-Mamba: Rotation-Invariant Mamba for Robust Text-to-Shape Retrieval
- 用全局与局部参考系分离几何与姿态,结合希尔伯特排序构建结构化点云序列
- 在超过200类物体上实现领先性能,任意方向下仍保持高精度检索
- 适合需要鲁棒3D检索的虚拟现实、游戏等场景
由于虚拟现实和游戏的普及,3D资产数量和多样性迅速增长,文本到形状的检索成为大型资源库中直观搜索的关键。然而,现有方法依赖标准姿态且仅支持少量物体类别,限制了其在真实世界中的应用,因为物体可能属于多种类别且呈现随机朝向。为此,我们提出RI-Mamba,首个用于点云的旋转不变状态空间模型。RI-Mamba通过定义全局与局部参考系,将姿态与几何解耦,并利用希尔伯特排序构建具有有意义几何结构的标记序列,同时保持旋转不变性。我们进一步提出一种新颖策略计算方位嵌入,并通过特征逐元素线性调制重新整合,有效恢复空间上下文并提升模型表达能力。该策略天然兼容状态空间模型,计算复杂度为线性。为扩展检索规模,采用自动三元组生成的跨模态对比学习,无需人工标注即可在多样化数据集上训练。大量实验表明,RI-Mamba具备卓越的表征能力和鲁棒性,在涵盖200多个物体类别的OmniObject3D基准上,任意朝向下均达到最先进性能。代码将发布于https://github.com/ndkhanh360/RI-Mamba.git。
原文摘要 · Abstract (English)
3D assets have rapidly expanded in quantity and diversity due to the growing popularity of virtual reality and gaming. As a result, text-to-shape retrieval has become essential in facilitating intuitive search within large repositories. However, existing methods require canonical poses and support few object categories, limiting their real-world applicability where objects can belong to diverse classes and appear in random orientations. To address this challenge, we propose RI-Mamba, the first rotation-invariant state-space model for point clouds. RI-Mamba defines global and local reference frames to disentangle pose from geometry and uses Hilbert sorting to construct token sequences with meaningful geometric structure while maintaining rotation invariance. We further introduce a novel strategy to compute orientational embeddings and reintegrate them via feature-wise linear modulation, effectively recovering spatial context and enhancing model expressiveness. Our strategy is inherently compatible with state-space models and operates in linear time. To scale up retrieval, we adopt cross-modal contrastive learning with automated triplet generation, allowing training on diverse datasets without manual annotation. Extensive experiments demonstrate RI-Mamba's superior representational capacity and robustness, achieving state-of-the-art performance on the OmniObject3D benchmark across more than 200 object categories under arbitrary orientations. Our code will be made available at https://github.com/ndkhanh360/RI-Mamba.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。