让机器人在场景变化时,靠记忆精准找物。
ChangingGrounding: 3D Visual Grounding in Changing Scenes
- 用记忆驱动策略,结合跨模态检索与多视角融合
- 零样本下定位准确率高,探索成本显著降低
- 适合真实机器人场景的持续感知与交互研究
现实世界中的机器人需根据自然语言指令在不断变化的环境中定位物体。现有3D视觉定位方法大多依赖重建并实时更新的点云,导致需频繁重扫,难以部署。本文提出ChangingGrounding,首个显式衡量智能体如何利用历史观测、仅在必要区域探索,并在动态场景中仍能精确输出3D边界框的基准。为此,我们提出Mem-ChangingGrounder,一种零样本方法:通过查询识别目标类别,检索相关记忆引导行动,高效探索目标区域;当先前操作失效时可回退,执行多视角扫描,融合多视角证据生成精准3D框。在ChangingGrounding上的实验表明,该方法在保持高定位精度的同时大幅降低探索开销。我们希望该基准与方法推动面向实际应用的、以记忆为中心的3DVG研究。
原文摘要 · Abstract (English)
Real-world robots localize objects from natural-language instructions while scenes around them keep changing. Yet most of the existing 3D visual grounding (3DVG) method still assumes a reconstructed and up-to-date point cloud, an assumption that forces costly re-scans and hinders deployment. We argue that 3DVG should be formulated as an active, memory-driven problem, and we introduce ChangingGrounding, the first benchmark that explicitly measures how well an agent can exploit past observations, explore only where needed, and still deliver precise 3D boxes in changing scenes. To set a strong reference point, we also propose Mem-ChangingGrounder, a zero-shot method for this task that marries cross-modal retrieval with lightweight multi-view fusion: it identifies the object type implied by the query, retrieves relevant memories to guide actions, then explores the target efficiently in the scene, falls back when previous operations are invalid, performs multi-view scanning of the target, and projects the fused evidence from multi-view scans to get accurate object bounding boxes. We evaluate different baselines on ChangingGrounding, and our Mem-ChangingGrounder achieves the highest localization accuracy while greatly reducing exploration cost. We hope this benchmark and method catalyze a shift toward practical, memory-centric 3DVG research for real-world applications. Project page: https://hm123450.github.io/CGB/ .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。