无需标定即可实现高精度3D定位，通过几何感知注意力提升语义一致性。
TrianguLang: Geometry-Aware Semantic Consensus for Pose-Free 3D Localization
- 引入几何感知语义注意力，利用预测几何信息抑制不一致匹配。
- 在五项基准上达到顶尖性能，单次文本查询取代多次点击。
- 每帧处理仅需57毫秒，适合实时交互机器人与AR应用。
从自然语言中定位3D空间中的物体和部件对机器人、AR和具身AI至关重要，但现有方法在场景优化的精度与几何一致性，以及前馈推理效率之间存在权衡。我们提出TrianguLang，一种无需推理时相机标定的前馈式3D定位框架。不同于以往独立处理视角的方法，我们引入几何感知语义注意力（GASA），利用预测几何信息门控跨视角特征对应关系，抑制语义合理但几何不一致的匹配，无需真实位姿。在包括ScanNet++和uCO3D在内的五个基准上验证，TrianguLang实现了领先的前馈式文本引导分割与定位性能，将用户操作从O(N)次点击减少至单次文本查询。模型以1008x1008分辨率每帧处理时间约57毫秒（约18 FPS），无需优化，适用于交互式机器人与AR的实际部署。代码与检查点见https://cwru-aism.github.io/triangulang/。
原文摘要 · Abstract (English)
Localizing objects and parts from natural language in 3D space is essential for robotics, AR, and embodied AI, yet existing methods face a trade-off between the accuracy and geometric consistency of per-scene optimization and the efficiency of feed-forward inference. We present TrianguLang, a feed-forward framework for 3D localization that requires no camera calibration at inference. Unlike prior methods that treat views independently, we introduce Geometry-Aware Semantic Attention (GASA), which utilizes predicted geometry to gate cross-view feature correspondence, suppressing semantically-plausible but geometrically-inconsistent matches without requiring ground-truth poses. Validated on five benchmarks including ScanNet++ and uCO3D, TrianguLang achieves state-of-the-art feed-forward text-guided segmentation and localization, reducing user effort from $O(N)$ clicks to a single text query. The model processes each frame at 1008x1008 resolution in $\sim$57ms ($\sim$18 FPS) without optimization, enabling practical deployment for interactive robotics and AR applications. Code and checkpoints are available at https://cwru-aism.github.io/triangulang/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。