用新视角合成技术完成3D物体补全，效果领先。
RaySt3R: Predicting Novel Depth Maps for Zero-Shot Object Completion
- 将3D补全转为新视角深度图预测任务，用Transformer模型处理光线查询。
- 在真实和合成数据集上，3D Chamfer距离最优，最高提升44%。
- 适合需要快速、准确3D补全的机器人与扩展现实应用。
3D形状补全是机器人、数字孪生和扩展现实的重要技术。尽管近期进展显著，现有方法仍存在3D不一致、计算开销大、难以捕捉锐利边界的问题。本文提出RaySt3R，将3D形状补全重新定义为新视角合成问题。给定单张RGB-D图像和一组查询光线（代表新视角），训练前馈Transformer模型预测这些光线对应的深度图、物体掩码和像素置信度。通过融合多视角预测结果，实现完整3D形状重建。我们在合成与真实数据集上评估，结果显示，RaySt3R在所有数据集上均达到最优性能，3D Chamfer距离相比基线最高提升44%。
原文摘要 · Abstract (English)
3D shape completion has broad applications in robotics, digital twin reconstruction, and extended reality (XR). Although recent advances in 3D object and scene completion have achieved impressive results, existing methods lack 3D consistency, are computationally expensive, and struggle to capture sharp object boundaries. Our work (RaySt3R) addresses these limitations by recasting 3D shape completion as a novel view synthesis problem. Specifically, given a single RGB-D image and a novel viewpoint (encoded as a collection of query rays), we train a feedforward transformer to predict depth maps, object masks, and per-pixel confidence scores for those query rays. RaySt3R fuses these predictions across multiple query views to reconstruct complete 3D shapes. We evaluate RaySt3R on synthetic and real-world datasets, and observe it achieves state-of-the-art performance, outperforming the baselines on all datasets by up to 44% in 3D chamfer distance. Project page: https://rayst3r.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。